16 ms·
Text editing hates you too (2019)
- Hackbraten 5y agoPlease add (2019) to the title.
- dang 5y agoAdded. Thanks!
- genpfault 5y agoPreviously: https://news.ycombinator.com/item?id=21384158 https://news.ycombinator.com/item?id=21384158
- floatingatoll 5y agoAlso “Text Rendering Hates You”, referenced by this post, previously: https://news.ycombinator.com/item?id=21105625 https://news.ycombinator.com/item?id=21105625
- bombcar 5y agoFor people involved with technology I think this is a perfect response to "how hard can it be" - no matter if they're discussing how to keep toilet paper stocked or why cURL can be replaced with five lines of shell script - it emphatically shows how complicated something apparently simple can be once you start diving into it.
- 7373737373 5y agoThis seems to be the ideal occasion to show off my latest pet project: https://icebergcharts.com https://icebergcharts.com
- LeifCarrotson 5y agoI appreciate that "making a ****ing iceberg chart" is at the bottom of the Hobbies iceberg :)
- shard 5y agoWasn't there a post or (most probably) a comment recently (within a month or so) with an iceberg chart? It looked very much like this site, but I can't find the right subject. Someone even commented that one of the best links in the chart was to a fictional story on the site itself. For the life of me I can't recall what the subject of the iceberg was, except that I think it was technology related, but I did spend several hours digging through all the links.
- shard 5y agoAh, found it in search. It was this one: The Cursed Computer Iceberg Meme https://suricrasia.online/iceberg/ https://suricrasia.online/iceberg/
- 7373737373 5y agoYep, that's where I got my inspiration (and some CSS) from, and I also link to it in the footer on my site: https://suricrasia.online/iceberg/ https://suricrasia.online/iceberg/ :)
- noofen 5y agoThis is fantastic. My only wish is to be able to export them as a PNG... maybe add a small watermark so people can find you.
- 7373737373 5y agoIf you scroll down there is "Download as Image" :) I'll try to create an unobtrusive watermark
- noofen 5y agoOh, I see. Perfect!
- slver 5y agoThings are as hard as we allow them to be. You know, fixed width ascii terminals in text mode sure were much easier in the 80s
- oblio 5y agoAnd then we discovered that not only English speakers want to use PCs.
- geenew 5y agoDidn’t extended ASCII cover that and make text terminals usable with non-English character sets? Genuinely curious.
- skrebbel 5y agoHow would you use extended-ascii to write Arabic?
- superjan 5y agoThe following was actually used: https://www.charset.org/charsets/iso-8859-6 https://www.charset.org/charsets/iso-8859-6 There’s no single byte encoding for Chinese, Japanese, or Korean though.
- enqk 5y agoIn practice, young folks write arabic with roman script when they chat. (with numbers to supplement the missing letters)
- qsort 5y agoExtended ASCII was actually several different and incompatible encoding, "codepages". It was kind of a solution, but there were major issues that made it much worse than Unicode/utf8: - Asian languages needed special encodings, 256 characters weren't sufficient. - It was impossible to work with different languages in the same document, each codepage only covered some character/alphabet. - There was no 'safe subset': Unicode codepoints 0-127 are exactly ASCII, which means that ASCII is compliant utf8. The same wasn't true at the time.
- tonyedgecombe 5y agoStrings, dates, floating point, even true and false are messy in some languages.
- colllectorof 5y agoMore often than not a problem that looks simple and has insane complexity once you dig a bit deeper is based on some faulty assumptions. Assumptions that either changed over time or never made sense to begin with. The wast majority of software systems out there are significantly simplifiable. However, not all developers are willing to analyze and question fundamental assumptions about systems they deal with. Most, in fact, are far more comfortable learning to jump through arbitrary hoops. They then brag about their "expertise" in the "domain" (which often doesn't even exist outside of software).
- leoc 5y agoAlan Perlis' old motivational-poster slogan "Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it." has a fair bit going for it.
- alex_smart 5y agoDoes that mean that Geniuses would be terrible software engineers, since removing complexity is seldom the right thing to do business-decision wise?
- edoceo 5y agoWhat? IME removing complexity removes costs. That's a good choice. I can't think of a case where it didn't happen. Of course this only works for the removable complexity, which is discovered through analysis (possibly iterative/recursive)
- alex_smart 5y agoTime spent removing complexity is time not spent developing features. And your users/clients care more about features than complexity (which is completely invisible to them) or even speed. They don't even care if your app runs 10 times slower than it needs to, as long as it runs at an acceptable speed. That is why software engineers are seldom rewarded at their companies for "removing complexity", only for finishing projects.
- cgriswald 5y agoI think there's some context that matters. I often don't need the full functionality of übertool and can implement the functionality easily because I don't have to worry about 90% of use cases with their associated edge cases. It's dead simple to keep toilet paper stocked in my house. But I would not use the method I use for my house to run inventory for a grocery store chain.
- shard 5y agoThat's exactly the point. The "how hard can this be" comments are often applying an individual use case, where corner cases can be ignored or worked around, to an enterprise situation, where corner cases may hit some legal constraints (e.g. accessibility) or may no longer be corner cases once in enterprise-scale (e.g. multiple languages / currencies / time zones).
- yourapostasy 5y agoI call this fractal requirements. It most commonly afflicts organizations with management that insists upon KISS interactions from their direct reports, but then micromanages the details when the going gets tough (scrutiny from higher layers). Some of the more common phrases you will hear in these interactions is, "I don't understand why this has to be so complex", or "Give me the 50 thousand meter/foot summary". They elevate their mental model of the domain to a level where they cannot resolve the granular details. Analogous to the lack of optical resolution of any telescopes (including Hubble) we have to see the American flag on the moon [1]. You need a very massive bandwidth (the telescope diameter) to resolve those details, but then you trade off for increased processing time (analogy breaks down here). Surprised Pikachu when they drill down, and the requirements explode in complexity. With most successful, highly-refined systems, it isn't the primary requirement that should be the focus of planning (Microsoft Word is just a text editor, how hard can it be!?). It is all of the 1% of features 1% of the market/stakeholders want that deliver 80% of the value proposition of the system to them. Stacked together, all those "corner cases" add up the value of the overall system, and add up the complexity. It isn't just a heap of features binned together; the team has to make sense of how it all fits together. [1] http://curious.astro.cornell.edu/about-us/45-our-solar-system/the-moon/the-moon-landings/122-are-there-telescopes-that-can-see-the-flag-and-lunar-rover-on-the-moon-beginner http://curious.astro.cornell.edu/about-us/45-our-solar-syste...
- dmitrygr 5y agoIn college we had a professor in software engineering class who had this wonderful saying: "no matter how complex a problem looks, if you look at it just right, you will see that it is in fact even more complex." It took years of working as a software engineer to begin to appreciate just how deep that is.
- deleted 5y ago[deleted]
- pood 5y agoI went to collage with this fella. He is a smart cookie. If you have not checked out his other blog posts they’re all p good.
- PaulHoule 5y agoYou know, every text editing system I've seen that knocks off word (has those "bold", "italic", etc.) selectors has something anomalous in how selections are handled.
- mihi 5y agoYes! So much this. I usually wish they'd just give me markdown or similar and only convert after I'm done. Slack is a prominent offender there.
- jll29 5y agoA good article, which shows that even plain text editing (without any formatty) is already way more complex than a layperson may appreciate. For the developer, there are some gems to be discovered when writing a text editor; recall that using regular expressions coverted to finite state machines saw the light as part of editing, and I remember being given a very cool paper (from the 1960s!) by my undergrad compiler construction professor that proposed an efficient algorithm for testing whether a string containing a line of text can possibly be a substring of any valid Pascal program (it was a piece of theory invented while and for a syntax-aware Pascal editor, possibly from IBM but not sure). Another thing is data structures: I learned of ropes, an alternative to strings for representing text (optimized for fast editing) in the context of text editing: https://dl.acm.org/doi/10.1002/spe.4380251203 https://dl.acm.org/doi/10.1002/spe.4380251203
- jmcphers 5y agoThere's a new(ish) editor called Xi that's built on the rope data structure: https://github.com/xi-editor/xi-editor https://github.com/xi-editor/xi-editor
- reificator 5y agoJust a heads up: That editor project, while conceptually interesting, is currently abandoned.
- Narishma 5y agoI don't think that's still actively developed.
- Oddskar 5y agoHaving touched adjacent technologies to this as well, I could not shake the feeling that text editing on the web, and WYSIWYG text editors especially, would be very well served by some sort of common understanding and agreement of output given certain inputs. As alluded to in the article, the devil really is in the details in this domain. When one knows what to look for, most rich text inputs really miss the mark in understanding intent and doing something reasonable on behalf of the users. An interesting example is the recent trend to start interpreting selection of text + pasting of a link as "turn this text into a link using the URL in the clipboard". It just makes complete sense. When would I ever want it to be "please delete the selection, and instead paste the URL"? Probably not very often. This is just one tiny example, but there are lots and lots of those in text editing. My gut feeling is a lot of the time this is all left to individual engineers to try to decipher the user intent; I would assume for the most part they either copy their peers or do a best effort guess. What if there was some sort of test suite one could run instead? Like a large number of test cases built upon interactions with actual users.
- breischl 5y ago>selection of text + pasting of a link as "turn this text into a link using the URL in the clipboard" Seems like an interesting UX debate. It's a handy shortcut, but it makes "paste" modal depending on what's in the clipboard. Pasting might replace the selected text, or might link it. And because the behavior depends on whether the clipboard text "looks like" a URL, there's an element of DWIM (Do What I Mean) in there. Does it only handle http:// http:// links? What about www.? Or mailto:? etc.
- Steltek 5y agoI haven't experienced the behavior described but reading it, I instantly hate it. It would differ from every other program I use. I'd counter with there should be two different paste actions in every program: plain and smart. I almost always don't want to preserve rich formatting when I paste text. It looks like my cat walked over the keyboard while I was typing. The action to paste as plain text versus a fancier/DWIMy method is often different from program to program.
- deckard1 5y agoRegarding the vertical cursor movement, back in the day there was an alternative behavior. You can see this in QuickBASIC/QBasic[1]. Here, Microsoft treated the entire screen as a grid. I always thought that was a great UI. It hides the fact that there are line endings (\n on Unix or \r\n for DOS). And why shouldn't it? That's an implementation detail. And as evident by DOS using CR+LF it goes back to the days where you had an actual physical carriage on a typewriter. Nearly every modern text editor you have to type out the spaces/tabs you want to reach the area you what to add text. In QBasic, you just move the cursor wherever you want on the screen with arrow keys and start typing. Because, unlike a typewriter, a computer screen is 2D. Vim has block mode but, AFAIK, can only work on selections. Of course, this is not universal and only works with monospace fonts in certain languages. But back in the day it made it easy to create ASCII/ANSI diagrams and art. Modern text editing is a multilayered mess. Rendering, input methods, variable width fonts, and the special codes (markdown, etc.). On some sites it's not even self-consistent. Take Jira. Last I checked, they have one editor for creating a ticket and an entirely different editor for editing that same ticket[2] (one of these editors sucks more than the other, but I can never remember which... but neither one is remotely what I would consider a "good" interface). How they got there is truly the 8th Wonder of the World. [1] https://archive.org/details/msdos_qbasic_megapack https://archive.org/details/msdos_qbasic_megapack [2] https://community.atlassian.com/t5/Jira-Software-questions/How-to-make-the-jira-text-editor-the-same-for-creating-and/qaq-p/1350071 https://community.atlassian.com/t5/Jira-Software-questions/H...
- qsort 5y ago> you have to type out the spaces/tabs you want to reach the area you what to add text If you move the cursor to an empty area and start typing, should the missing characters be tabs or spaces then? Or NULs? Also, how do you insert spaces/newlines/tabs in between other text? Everything would be implicitly modal. It's good UI for some things, but it's not as good for a general editor like vim. I'm not looking forward to write Python with it, for example :) IIRC the Commodore 64 did something similar with its built-in Basic prompt.
- deckard1 5y ago
- phkahler 5y agoOn the emoji color (skintone) modifier... A normal person wants to select their emoji from a set. Only a programmer would allow the user to input a tiny program with a symbol and a color modifier and then interpret it. If you want to make text editing a little easier, treat every character along with any style applied to it as a single entity. This way you avoid a lot of potential problems like in TFA as well as things like nesting of modifiers that may not be allowed to overlap - output those when all the characters are complete. In other words, don't track tags but generate them as a post-process when storing the data.
- akersten 5y agoYeah, the "Bad #2" example, how Chrome does it, makes perfect sense and seems correct to me. Modifiers get removed when the character they're modifying is removed. I would say leaving the modifier and applying it to the previous character would be certainly incorrect (unless you're in a hex editor, in which case combining the bytes as emoji at all is questionable). Imagine it were two different emojis and a skin color modifier after the last one (and somehow, through a rendering bug, the skin color modifier hadn't auto-combined) - OP's precedent for what happens when deleting the middle emoji would move the modifier to the first emoji! Clearly wrong, especially if it then decides to auto-combine. I mean, the example is pretty contrived in the first place: > Whoops, I didn't mean to put the a there. How in the world are you typing emoji such that you have to add the skin color modifier separately that would even afford an opportunity for an errant character to get inserted between? Windows numpad alt codes?
- wruza 5y agoI second that. Half of these “issues” arise from the fact that author is a programmer-y person and knows how unichars work. Normal people have no clue and completely no expectations on that matter, and don’t wanna have. Neither do I. If I’d fuck my emoji up in such way, I’d just expect to backspace it and type it from scratch. No need to spend mythical man-epochs on a trivia nobody really cares of.
- pcwalton 5y ago> Jonathan Blow, in a talk about how software is getting worse, points to the example of Ken Thompson's text editor, which Ken built in a single week. This kind of sentiment drives me crazy. Yes, internationalization makes things harder and more complex than they were in the old days. But, back then, only people who lived in one small part of the world were able to usefully use computers in their language. (Simply supporting the Unicode character set is not enough.) Nostalgia for the simplicity of the past ends up having ugly cultural implications. It's easy to say "let's go back to the time when things were simple"; it's a lot harder to say "folks in (e.g.) Israel shouldn't be able to type in their native language".
- hannasanarion 5y agoEspecially since the text editor in question is `ed`, which is not exactly a pinnacle of utility. We're really comparing apples to moon rockets here.
- qsort 5y ago`ed` was developed in 1969. That's a remarkably uncharitable take, considering `ed` itself and its derivatives (like `sed`) are still useful today.
- hannasanarion 5y agoNo more uncharitable than saying that modern developers suck at their jobs because they can't make Sublime Text in the same time as it took Ken Thompson to make ed.
- qsort 5y agoThat's not the point of the article, not even remotely.
- Groxx 5y agoThis is accurate in the same way as saying "a rock chiseled into a wheel with a stick as an axle is still useful today". Or maybe "a mechanical calculator and a book of logarithms is still useful today". Sure, it still exists and still does what it used to do, and its derivatives surround us. But nobody does that any more because nearly everything else is so superior, it'd be ridiculous to do so, except as an art piece or when you literally have no alternative.
- wdb 5y agoThese days I make sure I am away making a cup of tea when these kind of topics are discussed at work. After writing a syntax highlighting editor in Delphi and a rich text editor in Flash/AS3 I try to avoid it.
- djoldman 5y agoIt seems to me that one problem is the allowance of ligatures. Are there more than 2^32 possible ligature/character combinations? Not that it matters. We're stuck with it all.
- fallingfrog 5y agoI suppose the latex approach is one way to make some of this easier. The document is plaintext, just some letters snd symbols and all the formatting and fonts etc is specified explicitly through typed codes. So that solves the problem of invisible boldface characters and line breaks. I always thought it bothersome how latex worked but it does make some amount of sense to separate content from presentation. Doesn’t help with internationalization though.
- pcr910303 5y ago> This is why, on the web, text input and keypresses are separate events. Terminals conflate these two, causing problems. Yeah, so please don’t create terminal TUIs please. Very disturbing to use TUIs that don’t work with CJK languages at all. Please, please create GUIs instead of TUIs. (I’m not blaming programs reading from stdin and printing data to stdout, but the ones based on ncurse and fill the terminal screen.) Unfortunately the status quo on text editing didn’t improve since the last time this came up on HN. Unfortunately everything I said the last time[0] still applies… [0]: https://news.ycombinator.com/item?id=21384596 https://news.ycombinator.com/item?id=21384596
- Aeolun 5y agoBut uh, I create these UI’s for myself. Why would I consider CJK languages?
- jonnycomputer 5y agoElsewhere here on HN people are complaining about those scientists who can't seem to replicate their results. But here I'm reading that all those smart programmers can't even handle text selection right. So I guess I just want to say: things get complex, really fast, and a little humility goes a long long way.
- wruza 5y agoSmart people love to solve complexity, but they often forget that “how it should work” is a complex knowledge by itself and the more people know, (1) the less they trust an implementation at hand, and (2) the more likely they have a different opinion. Sometimes you just have to kick the can down the road and accept that nothing and nobody is ideal. Then you get a working system that everyone loves (except chinese).
- GekkePrutser 5y agoYeah text editing is hard. I've yet to find a Linux console editor that can do word wrapping smoothly yet preserve the normal line endings. Basically what the textbox on every GUI browser does right out of the box. It seems to be a very difficult ask.
- disgruntledphd2 5y agoIt shouldn't be I suppose. Personally, I've written so much in editors over the years that the common GUI behaviour seems weird to me.
- dhosek 5y agoI remember back in the early 90s wanting to write a custom IDE for LaTeX. Even in those 8-bit L-R text only monospaced font days, text editing was remarkably complex. Complex enough that I gave up before I started as I faced down the complications of being able to move from one line to the next.
- TwoBit 5y agoGood article but I would like to point out that I too looked into the Windows Text Services Framework years ago and it was literally the worst designed and implemented API I had ever worked with in my career. I got ahold of some engineers within Microsoft to ask wtf? and their response was that was written by some disconnected Asian office who understood Asian requirements well but were terrible architects.
- p2t2p 5y agoThat kind of post brings philosophical questions as well. Do we need to support 7000 languages? Should we? If we take a vision of United Earth a-la Star Trek, should there be support for any script other than “United Earth script?” Is in reasonable or unreasonable to expect countries/languages/people to upgrade their script to more simple/efficient things? Russian alphabet has had some number of simplifications and reductions throughout the history, can the same happen to Mandarin? Bunch of my Indian friends are more comfortable typing Hindi using Latin script rather than the native one. Not because of technical hurdles but because it’s cumbersome to apply all correct modifiers. In other words, should computers adapt to people here or should people adapt to computers or some kind of mutual movement towards each other with finding a middle ground? There’s a phonetic alphabet that contains all sounds from all languages (I don’t remember the name). It would be interesting to exercise a thought of a keyboard built to accommodate it and that alphabet serving as universal alphabet for the planet. Edit: typos
- iainmerrick 5y agoI think this overstates the general case a bit, although it has some great examples. Basic text editing is not inherently hard, as the people here citing “ed” point out. Even text editing left-to-right ASCII in mouse-driven WYSIWYG style is not too hard -- I’ve done it, for old Mac apps where there was no good reusable OS widget, and it was tough but tractable. What is incredibly hard is supporting editing in many different scripts. I think it’s hard for two reasons. First, there’s inherently just a lot of ground to cover -- multiple scripts, multiple encodings, multiple text directions, contextual rendering, emoji. Second, you can’t cleanly separate it into layers, as text input, text selection and text rendering all impact each other (I love the example of Chinese text input messing with vim’s command mode). If you could just find a way to keep those layers separate it would go back to tough-but-tractable. But from a user’s perspective, having the layers mixed up is (arguably) what makes it easy to use! You don’t need to switch back and forth between different modes to do stuff, it’s all just WYSIWYG. (Although what with emoji selector popups and the like it’s not actually perfectly modeless, like it can be in the left-to-right 8-bit encoding case.)
- slim 5y agoThe best support for bidi text is in gnome gedit. To the developers : Thank you
- layer8 5y ago> The emoji modifier is now modifying the newline character. In that case, the modifier should be displayed at the end of the preceding line, as newlines are generally associated with the line they are ending, not with the subsequent line.