6 ms·
I feel like no project should be allowed to advertise itself as "fast" unless it presents at least one (1) measurement demonstrating it.
by platinumrad 6y ago
I feel like no project should be allowed to advertise itself as "fast" unless it presents at least one (1) measurement demonstrating it.
- mhh__ 6y agoI'm slowly putting together a library and series of blog posts for really showing this, so stay tuned (one measurement often isn't enough). I've always wanted to be able to try and verify performance bounds at the CI stage, and I'm hesitantly optimistic it can be done cleanly (although amortized analysis is extremely important, you can actually play "guess the C++ std container from the graph" if you plot the data right)
- hsbauauvhabzb 6y agoPython compared to raw assembly development is slow as hell in computationally heavy scenarios. Development speed of applications in Python compared to raw assembly is nearly infinitely faster, both because it’s easier to write and because the talent pool for ASM is near zero in most cities. But the same can be said for basically all metrics people use to sell products.
- vincnetas 6y agoRunning subjective benchmark now. Generated 1G text file. Vim and nano after some delay are showing things on screen, ox is still thinking. 5 minutes and counting... base64 /dev/urandom | head -c 1000000000 > file.txt
- jeswin 6y agoYeah but that's not a realistic usecase for most users/developers. I haven't measured if Ox is faster or slower, but for me startup time and ability to quickly open files under 10MB are more important.
- saagarjha 6y agoI open massive files in a text editor all the time.
- bootcampwhere 6y agoI also am regularly limited in opening large text files working with genetic data.
- Aeolun 6y agoProbably, but then we don’t use our normal editors when we want to open huge files, we use the fast ones.
- vincnetas 6y agoBut for 10MB range do you even feel the delay for any of text editors? For me it feels instant whatever (terminal based) editor i'm using. And what does it even mean "fast" in this case. They all are definitely faster than me typing.
- fnord123 6y agoIt is very realistic when I copy a json dump into an editor and then do a key stroke to format it. My beloved vim doesn't fair so we'll. VScode does very well on this.
- ploxiln 6y agoI noticed some years ago that vim can be much slower due to highlighting of particular syntax of huge files. For example it was fine with a 10MiB C file, but super slow with a 10MiB XML file, but fast again with ":syntax off"
- _djo_ 6y agoAre you doing the formatting via a Vim plugin? I often find it's much faster to pipe out to a command line program to do formatting on huge files. e.g. :%! jq or :%! xmllint --format -
- OJFord 6y agoHuh, thanks, I didn't know you could put the contents on stdin like that (%!) - just in case you (or anyone) was similarly unaware in reverse, you can also put `%` wherever you want to use the filename in the command. i.e. I would have done: :! jq % and have probably been guilty of (not knowing `%!` per above): :! cat % | ...
- bruh2 6y agoI frequently open minified CSS/JS by accident. IntelliJ's project search window includes a small preview of each result when you hover it, so when you navigate through the results your might pass through a minified file and get your IDE to hang for a couple of minutes :)
- ginko 6y agoI open massive text files semi-regularly. If a text editor can't deal with that it's a big con for me.
- pessimizer 6y ago> Yeah but that's not a realistic usecase for most users/developers. It also looks like vim and emacs don't decide what a realistic use case is, or if you're "most" users/developers. They just open text files for editing.
- beowulfey 6y agoIs something like Emacs, Vim, or Nano genuinely slow for you? Even for large files, I've never noticed anything beyond microsecond-type delays with those. Definitely does not feel laggy to me, usually.
- OskarS 6y agoOne thing about this particular benchmark is that the file it produces is a little bit unusual even for large text files, because it is only contains one (massive) line. Most huge text files you open in text editors are things like multi-gigabyte logs and database dumps, which are composed of many relatively short lines. Text editors often bake that assumption into their data structures. A more representative test would probably be to use hexdump instead of base64 there. Though I take the point: if a text editor advertises itself as being "fast", this is the kind of stuff you expect it to handle. EDIT: ok, so i just tried the hexdump version. Vim and Emacs took a couple of seconds each (Emacs being slightly slower and chunkier when you navigate, Vim handles it without issue and is perfectly repsonsive). less opened it instantly (obviously, it's less!). ox has been going for a few minutes now, nothing happening. It's not the benchmark's fault, ox is just slow for large files.
- josefx 6y ago> the file it produces is a little bit unusual even for large text files, because it is only contains one (massive) line. XML and JSON without pretty print. Minimized HTML with embedded resources. While not often gigabyte sized those seem to be enough to get most editors to hang and make editing the files a miserable experience.
- OskarS 6y agoYes, true enough, it does happen occasionally, but it's pretty rare (and usually there are other ways to deal with it: using less or similar tools, or just running your massive single-line JSON thing through jq first). And my point was exactly what you're saying: many text editors choke on files like this, so it's not such a huge indictment of ox that it chokes on it as well. However, ox clearly performs poorly even when your text files aren't massive single lines, so it doesn't really matter: ox basically can't handle files like this regardless of line distribution.
- vram22 6y ago> and usually there are other ways to deal with it: using less or similar tools, or just running your massive single-line JSON thing through jq first Older Unixen at least (SVR3-based) had a tool called bfs - big file scanner. Used it some, e.g. when, as a system engineer, I helped IT staff of one of our customers, a university processing exam results of tens of thousands of students from dozens of colleges affiliated to that univ. IIRC, its UI was something like a read-only ed (google "unix bfs command"). You used it to scan really large (for the time) files, e.g. data files (input/output) in large data processing environments, for purposes like checking if the input or output files anecdotally looked okay, no major noticeable garbage in them. Haven't checked if it is present in modern Linuxes. Also don't remember if it could handle large files without newlines. Likely not, if based on ed. Didn't have such files to work on then.
- alkonaut 6y agoMost would happily trade an editor that is 100x slower for gigabyte single-line text for one that is 10% faster on gigabyte multi line text.
- Cthulhu_ 6y ago99% of files I work with are <1000 lines, so I don't mind. For larger files I will fall back to Vim. The important thing to realize is that text editor performance is less about the technology used and more about the data structures and i/o.
- tastyminerals2 6y agoThis is not an excuse for an editor being claimed "fast". Textadept is written in C and is also very fast but has the exact same issue -- choking on huge files when Vim and others are able to handle it.
- weinzierl 6y agoI rarely open gigabyte single-line files, but when I do, I don't want to switch editor. Most of the time I only realize that everything is on a single line after opening the file. A good editor doesn't require me to think about these things.
- ajuc 6y agoIf it's over 1 GB and single line you're almost always going to exit the editor, run some scripts on it, and open it again when it's correctly formatted.
- levosmetalo 6y agoI would rather open it in editor, strike a "reformat all" shortcut, and have the file right there. I don't want to chase separate tools that can format different text files, that's what editors are for.
- 6y ago
- matt-attack 6y agoEmacs opened it in about 4 secs on my 2018 MBP.
- rootlocus 6y agoI highly doubt you have to manually edit a GiB file by hand. For read-only, less opens it instantly.
- the5avage 6y agoVim is amazing. Try searching for patterns in this enormous file ;) Definitely a hard nut to crack. Personally i see no reason why this is needed (just use vim), but I'm sure a lot of people will use it just because its written in rust^^
- deleted 6y ago[deleted]
- bayindirh 6y agoSoftware developers find lazy/partial file loading unnecessary but, it is not and it won't...
- politician 6y agoSummary of replies: This benchmark doesn't count because No true Scotsman.
- zelphirkalt 6y agoI wonder: Couldn't we have some kind of simulated line breaks, which an editor places virtually, so that it can manage very long lines easily and efficiently internally, while writing out the file without these virtual line breaks? Basically only so make the conditions appropriate for the editor's string data structure?
- dahfizz 6y agoIn fact, you basically need to do that anyway to wrap long lines effectively.
- makach 6y agoIt runs in the terminal. I think most people assumes that terminal === fast.
- silon42 6y agoDirect to X can be much faster, as there is no middle man. And can have better keyboard (and mouse) support which in terminals is pretty poor. This is why vi(m) is best for terminals, because it's keyboard shortcuts are adapted for that (except typical 1 second delay on escape key).
- lopatin 6y agoBut it's close to metal \s
- 1023bytes 6y agoThe input is actually very slow. It's own configuration file (about 300 lines) took over a minute to paste.