4 ms·
Apart from the learning experience, there is still value in a small-sized, fast TUI text editor for Linux. However, my two suggestions would be: - add Unicode
by jll29 2y ago
Apart from the learning experience, there is still value in a small-sized, fast TUI text editor for Linux.
However, my two suggestions would be:
- add Unicode support (it's the 21st century, so ∑, ®, etc. should finally work); this is not easy, but the earlier you do it the less dramatic the changes will be (check out https://github.com/unicode-org/icu/tree/main/icu4c); https://github.com/unicode-org/icu/tree/main/icu4c);
- don't allocate the lines individually (as you do now with malloc, having looked at the code); at least use an arena allocator, i.e. allocate larger chunks of memory and the provide your own alloc_line function that uses the larger chunks (called arenas) piecemeal. This will speed things up and reduce fragmentation. A more advanced approach would be not to use per-line buffers but to switch to rope data structures e.g. https://github.com/josephg/librope https://github.com/josephg/librope).
- badsectoracula 2y ago> This will speed things up and reduce fragmentation Does this really make much of a practical difference these last decades? I wrote a text editor back in the late 90s in DOS using DJGPP with per-line allocation and it worked fine on the Pentium MMX i had at the time (and my approach to optimization at that time was be ignorant of the concept :-P), i'm not sure it'd really make much of a difference on any desktop or laptop CPU released past 2000 - at least as far as users editing text files are concerned (might make a difference if you try to open a multiGB file with barely a line break and accidentally press a letter :-P).
- HexDecOctBin 2y agoNo, it won't. You might want an arena for other reasons (better reasoning about lifetimes, preventing nest of pointers, etc.), but an array of line array approach to text editors is totally valid (except in the case of millions of extremely small lines).
- lanstin 2y agoI do from time to time open files of millions of small lines. To this day vim handles it better than emacs. These would be log files.
- HexDecOctBin 2y agoLog files have lines that are long enough, I was thinking more of one/two-character long lines (such that the overhead of malloc bookkeeping would be more than the line size in memory). Regardless, Emacs also stores the data in a single array with a gap in middle (and memcpy is relatively cheap), so what slows it down is probably not the choice of data structure but everything else built around it.
- lanstin 2y agoYeah I have very high limits for paren matching and syntax highlighting so probably those scanners.
- simjnd 2y agoAs an alternative to ICU, there is suckless's libgrapheme (https://libs.suckless.org/libgrapheme/ https://libs.suckless.org/libgrapheme/) which is more than a 100x smaller and provides full Unicode compatibility.
- wwalexander 2y agoHeh, I maintain the MacPorts port for this lib! Like all suckless projects, it’s written in very simple and portable C which makes it a breeze to package. https://ports.macports.org/port/libgrapheme/ https://ports.macports.org/port/libgrapheme/
- frign 2y agoThanks for your work on packaging my library. Please let me know if I can make the process simpler for you; I take great care to make packaging as simple as possible for the packagers. Likewise I have no sympathy for those writing software that is almost deliberately hard to package.
- wwalexander 2y agoThank you for your work on the library! It really couldn’t be easier to package but thank you <3
- HexDecOctBin 2y agoAre there any similar options that handle collation and normalisation? No small Unicode library seems to implement them, unfortunately.
- frign 2y agoAuthor of libgrapheme here: Both collation and normalisation are non-trivial and have many gotchas thanks to the way the Unicode consortium likes to write their specifications. I sometimes get the feeling that they don't even care about implementers and just document what is done in the reference implementation ICU. The only sensible normalisation one can implement is the full decomposition (NFD), and maybe the full composition (NFC). You rely on the full decomposition if you want to collate correctly, which is a problem because the amount of memory needed to store the decomposition is unbounded in general. I don't want to make the libgrapheme users jump through hoops, and I also don't want to do any memory allocations in libgrapheme either. There is an idea floating in my head on how to solve this, but I'm currently busy finalising Unicode 15.1 support (Unicode 16.0, released on the 10th, will be trivial to upgrade to) and releasing my already fully-compliant implementation of the Unicode bidirectional algorithm.
- wolfspider 2y agoWell yeah, Seph is right as usual I’ve been updating my MLIR generation code to use Twine lately: https://llvm.org/doxygen/classllvm_1_1Twine.html#details https://llvm.org/doxygen/classllvm_1_1Twine.html#details LLVM has been using rope data structures for some time. There is more discussion about it here: https://lists.llvm.org/pipermail/llvm-dev/2011-July/041639.html https://lists.llvm.org/pipermail/llvm-dev/2011-July/041639.h... In my situation when parsing lines of code Twine stores the file path and concats the line number and column when getting the location within a file. It does this a lot so using Twine there really helps. The rope data structure can be used in a number of ways.
- velorek 2y agoI agree. Unicode support is top priority. Thank you for sharing those links.