11 ms·

Simple software things that are actually very complicated

167 points by AshleysBrain 4y ago

PaulHoule 4y ago
It's a funny topic for me because last summer I wrote a text layout engine for Python because I thought the alternatives sucked. I struggled with loading libraries to support vertically-oriented Japanese text and then it hit me "these characters are all squares... it can't be that hard!" Then there was the gradual epiphany that: (1) if you want to use serif typefaces and have it look good you have to kern them properly (2) Word, Powerpoint, and Adobe tools don't kern properly (3) that's why sans serif typefaces are so fashionable these days Of course I am using this to print cards with custom software and it doesn't have to be interactive, reflow, or be useful to anyone else.
- froh 4y ago
  Yes, and also ligatures, serif fonts love ligatures...
  - PaulHoule 4y ago
    Another complaint I have is that if you look really close at most typefaces the letters and numbers seem to not just not come from the same typeface but not even from the same planet.
  - dhosek 4y ago
    Which are (a) not unique to serifed faces (the original Avant-Garde famously was full of discretionary ligatures) and not necessarily ideal for serifed faces. Many typefaces designed for Linotype hot metal look decidedly worse with ligatures than without because the f could not have any overhang so if you had something like fifty, the two fs looked bizarre in conjunction. My go-to type for demonstrating this problem was Trump Mediaeval but sometime in the transition from Type 1 to Opentype, the type gnomes at Linotype saw the obvious issue here and replaced the ugly fi ligature with a logotype with fi without the ligature (the logotype existing more for the sake of documents which might have manually specified ﬁ rather than fi (I’m curious to see whether the difference is visible when this gets translated into its presentation form)).
- dhosek 4y ago
  Here’s the secret of typesetting vertical Japanese text: You turn all the characters anticlockwise 90°. It is now exactly the same process as typesetting any L-R text. Regarding your epiphany, (a) Word doesn't turn on kerning by default, but it does indeed kern properly. I don't use Powerpoint, but I'm guessing there might be a similar setting on the second tab of the font modal for turning on kerning. I don't know where you get your claim that Adobe tools don't kern properly, but they very much do. And finally, kerning is just as important for sans-serif type as it is for serifed type. The only arena it doesn't come into play is monospace type. Which, perhaps surprisingly, Japanese type traditionally is.
userbinator 4y ago
I'll offer the counterpoint that it's only complicated if you want to make it so. If you only need e.g. monospace ASCII, that doesn't take much. Do you really need proportional fonts? Vertical layouts? Keming? In my experience, a lot of the time the answer is actually "no" --- and thus the simpler (and more efficient, less bug-prone) algorithm works just fine.
- zokier 4y ago
  Do you really need more than 1s and 0s? Maybe go back to blinkenlights and toggle switches, those are simpler and even less bug prone to implement. You can still perform any computation you want, but it sure will not be as pleasant as having nice ux.
- FooBarWidget 4y ago
  Whether you need them is up to users. If you have Asian users then ASCII will not do. No kerning? Why would users put up with ugly text rendering? Just to make our lives as developers easier? This attitude reminds me of what someone says about the difference between German and Japanese tech. Japanese ones last forever because they overdesign their products to avoid breaking even when misused, because they are customer-focused. Germans are rule-focused, so they design their products according to spec. German products also last forever — if you use the product exactly per instructions. But if you make a mistake... good luck.
  - userbinator 4y ago
    "ugly" is entirely subjective.
    - can16358p 4y ago
      Well, if 90 out of 100 users think it's ugly, that would better be resolved to have more users use your library, even if you are in the 10% as the developer.
  - eska 4y ago
    As a German working in a Japanese company this kind of romanticization of these two cultures always makes me snort.
EdSchouten 4y ago
Parsing and displaying a floating point number, a la strtod() and printf(“%f”). I think that’s always the prime example of something that looks simple, but simply isn’t.
zokier 4y ago
Are there any simple software things that are actually very simple? I feel like tons of the very basic things that have been core software tasks for ages still have surprising amount of complexity and depth to them when you'd expect them to be simple. Stuff like text (basically anything), math (real numbers), 2d graphics (efficient high-quality vector graphics still seem like a pipedream), sorting (we just had a thread about improving sorting in postgres!), files (infamous fsync issues), color, etc just seem to constantly come up as problems when you'd think we had solved them already 20 years ago.
- syntheweave 4y ago
  Most software things become simple when you can define them down into an enumeration of possibilities. For example, if my text problem were "display these preformatted strings in a single resolution and layout" there would be no requirement on their encoding or processing and I could do whatever I like, even hardcode them or store them as image assets. Games have done this trick since forever because they have a tremendous capacity to soak up more and more static content, and much of the core tech in a game engine is in finding the ideal representations to efficiently load and present that content while carefully minimizing the dynamic behavior that would entail a general-purpose solution. The problem for software in the larger view of things is always that we've defined the problem with sufficient generality that it has to accommodate all the depth, because we don't know who is using the software or what they want from it. The medium is never "set in stone" so the tools are forever adapting to a new use case. And just when you think you've modelled every possible aspect of the data, some other way of doing it will come up. And it's not a right thing either to think "OK, I'll just solve the most general thing I can think of now so that I don't deal with it later" because then you just have a wide field of untested edge cases and no coverage of the thing you didn't anticipate. Like, take image editing software as an example. They all let you draw things freehand. Many support pressure sensitive stylus input. But the way they interpret that input is all over the map: different sensitivity curves, stabilization algorithms, brush behaviors and so on. There's no winning the battle by defining the most general engine for freehand drawing and painting, because what the user craves most of all here is the path to "just works" defaults. Thus in every sufficiently developed editor, an enumeration of possibilities appears again, but as a configuration letting you browse presets.
tester756 4y ago
http://johnsalvatier.org/blog/2017/reality-has-a-surprising-amount-of-detail http://johnsalvatier.org/blog/2017/reality-has-a-surprising-...
umvi 4y ago
Some of these things are only super hard if you are developing a general solution. The general solution to word wrapping across all written human languages in use on the planet is indeed difficult. But implementing basic word wrap for an HTML5 game that only English players will play isn't too hard.
- bo0tzz 4y ago
  Unless you're targeting a very localized audience (eg employees in one company, kids in one school district), I don't think it's safe to assume that only English users will use your software.
  - Aeolun 4y ago
    It may not be safe to assume only English speakers will use your software, but it is certainly reasonable to support only them.
  - adrianN 4y ago
    If your software only supports English, most of its users will be fine with English.
ivraatiems 4y ago
I agree with the general premise - sometimes "simple" things in software are really complex - but not the specific examples. The reason for the complexity in the author's examples is that they're using web canvases in a way they are not, as far as I know, intended to be used. Canvases are for 2D shapes/bitmap rendering, not complex text and user interactions. You're having trouble because you're reinventing the wheel. Maybe it's necessary for your use case, but that doesn't make it shocking that it's hard.
- ec109685 4y ago
  Unless you’ve done it before, there’s no way you would think it to be that complex to simulate the OS behavior. That was the point of the article. When reading that list, my thoughts turned to whether there is a good open source implementation that could be used / cross compiled into web assembly since surely somebody has done it correctly.
kccqzy 4y ago
Somewhat famously, Knuth came up with a dynamic programming algorithm for breaking paragraphs into lines with full justification (both left aligned and right aligned). Later on there are of course further innovations like character protrusion and expansion to achieve that optically aligned look without much (if any) hyphenation.
- PaulDavisThe1st 4y ago
  I am not sure it's clear that Knuth's work is appropriate for all or even any non-Roman alphabetic languages.
  - svat 4y ago
    The best typeset books I've seen in Devanagari (a non-Roman script, used for Sanskrit, Hindi, Marathi, Nepali) have been typeset with XeTeX (XeLaTeX). Of course this involves other system libraries like Harfbuzz to handle the kerning and measure widths of individual words, and leaves some of the complexities mentioned in the post to the user, but the Knuth–Plass algorithm from TeX is still what is at the heart of it, for breaking paragraphs into individual lines. Take a look at TUGboat and the TUG conferences or the TeX.SE website to see it being used for many languages and scripts (including Arabic, Hebrew, etc). TeX is also big in Japan it appears, and they maintain their own variant engines (ptex, eptex, uptex, euptex).
    - PaulDavisThe1st 4y ago
      In the 80's, I did the typography design for two books with TeX and LaTeX, so it's not as if I'm unfamiliar with the stuff :) However, it is true that I haven't seen many examples of non-Roman script being well-set with TeX, so I guess I should take another look.
  - cracrecry 4y ago
    It was used exactly for that application. Not that long ago, typesetting math formulas was extremely expensive, unless you used TeX. It was not very commercial and companies were not interested but certain individuals were. The same happened when you wanted your Hebrew or Arabic or Japanese just to look right. Japanese uses at least four scripts plus latin words in the same place. Over time, commercial entities added support for those, most of them just incorporating TeX source code (as it was public domain). They added the visual interfaces and the easiness of use that TeX lacks.
trhway 4y ago
I think the complexity we handle has stayed the same. It is just for the same amount of complexity and mental energy we get more functionality these days than decades ago. Text is a good example - the same complexity and the same amount of code handles these days a huge number of languages whereis decades ago that complexity would give you only one.
nraynaud 4y ago
I had a client simultaneously telling me he wanted a function, but that it couldn’t take that much time to develop it because it’s only useful 5% of the time. Sorry dude it’s all or nothing, it’s not like mechanics were there is a range of solutions going from sheet metal to forged titanium or from a piano hinge to magnetic bearings. I’m a bit unfair, because some high tech stuff is cheap, we all use tree and hash structures whose development we don’t pay because they are in the standard library, but it’s a few strategic and standardized places. The same as using a ball bearing in mechanics: you get into a very high tolerance world for cheap, the mechanical interface is standard and some unicorn mounted helve make those bearing in a chocolate factory as far as I know they might develop maps and queues on their other half time.
Gigachad 4y ago
A lot of the complexity in this article isn't so much that the task is very complex, but that if you want to support every single language, culture, format, then it becomes a very long task. I do wonder if we will end up just moving towards more unified formats and doing away with the need for all this complexity. I could be very wrong on this, but it seems like for the smaller to mid scale software projects, I18n is just not worth it. Throwing a bunch of YAML files over the wall to a translation company and then just dropping them in the app does not work and the end result is so bad that native speakers would rather just use the app in correct English than broken native language. Now you don't have to care about how to wrap or hyphenate a word in an obscure language which doesn't support it, and you don't have to worry about vertical text. Your problem space just got a lot smaller. And if you are building a tool at Google scale then you should have the resources to just make the problem go away anyway.
- colordrops 4y ago
  Supporting "every single language, culture, format" *is" the task in the article. You can decide whether you want to do it or not, but that's a different question.
  - AnimalMuppet 4y ago
    But who thinks that's simple? That's an insanely complex task.
    - Gigachad 4y ago
      At a high level the task may seem simple "Wrap the text when it reaches the end of the line". And if you only care about English, it actually is reasonably simple. The complexity is stuff you wouldn't even think about unless you had experience in the area.
      - AnimalMuppet 4y ago
        Fair, to a point. But colordrops said "supporting every single language, culture, format". So by definition, it's more than English. And as soon as you start looking into non-English writing systems, you realize that it is very much not simple.
PaulDavisThe1st 4y ago
I found it amusing to read this, given that in the context of Ardour (0) we still have the long term stretch goal of replacing GTK with an expansion of our home-grown canvas-based widget system. There are 5 widgets missing, one of them is the text entry. The description in TFA regarding doing this in the context of an HTML canvas element is more less identical to the issues that exist for a native re-implementation, and there's consequently a similar high pressure rationale for giving up on the idea. (0) https://ardour.org/ https://ardour.org/
- robocat 4y ago
  https://workspaceupdates.googleblog.com/2021/05/Google-Docs-Canvas-Based-Rendering-Update.html https://workspaceupdates.googleblog.com/2021/05/Google-Docs-... mentions some of the other complications like assistive screen-readers, extensions, and how add-ons work without HTML. https://docs.google.com/document/d/1vUnn0ePU-ynArE1OdxyEHXR2G0sl74ja_st_4OOzlgE/preview https://docs.google.com/document/d/1vUnn0ePU-ynArE1OdxyEHXR2... feels very weird on an iPad from a quick 10 second look: selection markers are Android styled, scrolling is janky, scroll easing is wrong, no scrollbar. Popups menus are very different and they have Mac keyboard shortcuts! Etcetera etcetera, and that was on a readonly document. (I presume it is the current version of Google Docs from a browser).
ronnier 4y ago
I've come to the conclusion that nothing is simple. Nothing. It's only "simple" because thousands of years of work were applied to get us to where we are today and we are building on top of that. It's arrogant to think anything is simple. Just to build a key cap for this laptop I'm typing on, very difficult if I actually had to do that. I need to use tools that were ultimately kicked off by stone age men thousands of years ago that progressed by building tools with those tools until we go to the point today where we can create a key cap for a laptop.
- mrwh 4y ago
  I'm sure you know this story already, but nice example: https://gizmodo.com/one-mans-nearly-impossible-quest-to-make-a-toaster-from-5794368 https://gizmodo.com/one-mans-nearly-impossible-quest-to-make...
  - ghgr 4y ago
    Thanks for sharing! It brought back to my mind this project of mine (more like a "wish" than an actual project) to build the current technology tree (like Age of Empire's or Factorio's) as a sort of wiki. And of course the wonderful essay by Leonard E. Read "I, Pencil".
  - amelius 4y ago
    A better example would be someone sitting in the middle of a jungle with no access to technology trying to make a toaster from scratch.
  - deleted 4y ago
    [deleted]
- dwohnitmok 4y ago
  Oh it goes even further than that, all the way to the Big Bang. As Carl Sagan said, “If you wish to make an apple pie from scratch, you must first invent the universe.”
- meribold 4y ago
  > I've come to the conclusion that nothing is simple. Surely it's more complicated than that ;)
jimbob45 4y ago
Browser text isn’t measurable AFAIK at least in any consistent way. I tried doing it for a project at work and came to the conclusion that it simply wasn’t possible. The SO answer below appears to agree with me. https://stackoverflow.com/a/12469203 https://stackoverflow.com/a/12469203
joshvm 4y ago
There are plenty of variations on this, I'm surprised no one has mentioned Falsehoods programmers believe about names yet: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... (as someone with a long, double barreled surname this is relatable) The common thread is usually what assumptions you can make about your use case, when are they valid and when do they fail?
- raphlinus 4y ago
  I think of "text rendering hates you"[1] as the classic "falsehoods" analog about text. [1]: https://gankra.github.io/blah/text-hates-you/ https://gankra.github.io/blah/text-hates-you/
buescher 4y ago
A whole lot of the simple functions in the standard C library are in there because a naive implementation will have bugs, often subtle ones. Implementing strcpy or strlen used to be a common interview question.
ozzythecat 4y ago
There are two things I absolutely do not want to implement, ever, if I can get away with it. Any kind of dynamic page rendering: we want to write code once and re-use everywhere, in the Web browser, mobile, iOS, Android, etc. This is a fool’s errand 99% of the time. You’ll end up building something super constrained if it is usable at all. Admittedly there are some viable options for this like React Native. Yeah, I just don’t want to work on this type of code. Client libraries: I want to write my own client, in top of a third party networking library, to send HTTP requests, and handle threading, tcp connections, connection pools, thread pools, etc. Even when using something as simple as libwebcurl, innocuous things fail in weird, unexpected ways that almost never come up in (my) unit tests, integration tests, and chaos tests. Server side client code that calls another service can be dangerous. Device code that you ship out to millions of devices is even more dangerous. Oh… 50% error rate because suddenly the code can’t access an SSL cert on the device. Wait… what??? I can’t even reproduce it? I’ve been writing code since grade school and am a decade now into my career. I’m an above average programmer on a good day, and terribly average at best on most days. So of course, I might be sounding melodramatic here.
gernb 4y ago
Now if flutter would learn all of that and stop trying (and failing) reproduce it! Guess what? OS Level and browser level spell checking doesn't work in Flutter because it's rendering pixels instead of OS text input / HTML text input. And a ton of other issues that will likely never be fixed. It almost makes we wonder if Flutter's goal isn't good UX but rather user control. If it's just pixels it's much harder for users to copy and paste, quote, scrape, etc....
- ec109685 4y ago
  Interesting they don’t use hidden native elements to simulate that. I guess kerning and layout would be subtly different between platforms. I wonder what Google Docs does on iOS? ReactNative doesn’t have this problem since the ui is expressed ultimately as native components.
dhosek 4y ago
Grapheme segmentation in Unicode is actually kind of interesting and completely data-driven. I wrote my own implementation of the segmentation algorithm in Rust recently¹ (because the existing segmentation crate didn’t have an interface that let me get the next grapheme cluster from a char iterator). It was fascinating to dig into the specifications and see how everything fits together. ⸻ 1. It will be published to crates.io sometime in the next month or so, there’s possible optimization work to be done, or at least, an alternate interface to the iterator to see whether the speed penalty I’m paying is an algorithmic problem or just an artifact of the trade-offs I have for my use case. I also ended up writing my own character class code as well and in benchmarking against the current popular crate got a 10x speedup,² so even if my segmentation code is useless to anyone but myself, I think that this code at least will be useful. 2. The other crate has a really inefficient implementation of looking up character classes where it ends up having to do a binary search in the data tables for many characters while I use a two-step table approach which lets me get a character’s class in O(1) time (and the tables are designed to be small enough that they’ll stick around in the CPU cache for lookups in a tight loop).
- kwantam 4y ago
  Sounds like a great improvement! Would it be possible to contribute your code to the existing crate rather than release your own? Then we could all enjoy the benefits on the next recompile :)
  - dhosek 4y ago
    The existing crate seems to have lost its maintainer.
deathanatos 4y ago
No, text on a canvas is a somewhat obvious nightmare. No, we must go simpler. left-pad: you need to account for wide characters, such as CJK characters but also emoji, you need to account for things like combining characters, etc. left-pad is not some trivial function. Of course, the infamous left-pad package was widely mocked "why do you need a whole package for such a simple task" — so simple, indeed, that it failed to handle many of those cases. But don't worry, it's been standardized now as padStart() … and still fails those cases: > 'e\u0301'.padStart(2) 'é' Okay, strings aren't simple! But numbers, numbers are fine right? So then, Go issue #4594, "add a Round function [to Go]"[1]. Go maintainer Ross Cox suggests that "it's a one liner" and suggests … a buggy implementation. There are a total of no less than 6 different implementations in the issue, and every single one of them is buggy. The bug is closed, the function is "too simple" to add. Thankfully, the wound gets reopened nearly a half-decade later in #20100. Another buggy implementation is suggested in that thread, and thankfully, Go gets a round function. (A non-buggy one … I think.) [1]: https://github.com/golang/go/issues/4594 https://github.com/golang/go/issues/4594
- mro_name 4y ago
  the basics are incredibly hard. Listen to a physicist explaining time, matter or energy.
  - deleted 4y ago
    [deleted]
- gernb 4y ago
  The entire idea of padding by characters only fits a few use cases. leftPad would never have covered those cases. If you want tabular data use a table or grid. No amount of string gymnastics will get you aligned strings across languages and fonts. So yea, leftPad was always a bad idea because the only useful implementation is too trivial too have been put in a 3rd party library. Unless you believe there should have been a library called 'add' that was just `export const add = (a, b) => a + b;`
orangepurple 4y ago
I was going to nominate mutexes and semaphores
lelanthran 4y ago
I don't know of any developer that thinks text rendering is "simple". Even some non-devs, due to using word-processors that completely "mess-up" everything the minute a single table is added, won't think that text rendering is simple. The best "simple-is-very-complicated" essay to read is "Reality has a surprising amount of detail" - http://johnsalvatier.org/blog/2017/reality-has-a-surprising-amount-of-detail http://johnsalvatier.org/blog/2017/reality-has-a-surprising-...
- ec109685 4y ago
  That was an excellent blogpost. This whole discussion reminds me of Joel’s Single Biggest mistake software developers do: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-... Namely, rewriting software is hard because the years of subtle complexity aren’t immediately apparent and hard to get right the second time (at least harder than first thought).
magic_hamster 4y ago
You got to leave out implementing semantic considerations when starting out. I.e. if your line break changes the meaning of a word, or using hyphens is not allowed in certain words for some languages, those are not technical challenges. They are semantic challenges and solving them will require heuristics or specialized handling. Although being aware of these considerations is helpful for designing a solution which will be able to address these issues later. Like determining allowed break spots before doing the actual line break. And having that logic extensible.
ycuser2 4y ago
Unrelated to the content: What is the reason to link to external websites like this: https://www.construct.net/out?u=https%3a%2f%2fen.wikipedia.org%2fwiki%2fInput_method https://www.construct.net/out?u=https%3a%2f%2fen.wikipedia.o... Why not set a link to wikipedia directly?
- leksak 4y ago
  To track if people clicked on the link maybe?
bribri 4y ago
There are many hidden complexities in seemingly simple features like text input fields and wrapping text in a canvas. It is better to let the browser or operating system handle these features, as they have been perfected over many years and are much more likely to work correctly than a custom implementation.
bullen 4y ago
I'm doing .ttf OpenGL with custom row break for my MMO: The trick is to take as much advantage of this command-line and remove all other GUI. I'm only going to support ASCII which helps a great deal. Chat bubble / command-line above characters is the only GUI the game will have: Login: /sign name pass Register: /join name pass pass Sensitivity: /sens 0.6 etc.
dhosek 4y ago
>Splitting strings to graphemes is actually quite slow, so you can't do it too often during word wrap, or you'll tank performance. If you're splitting strings to graphemes for word wrap, you're doing it wrong. Unicode segmentation defines a word segmentation algorithm that should be used for that use case instead.

167 points by AshleysBrain 4y ago

It's a funny topic for me because last summer I wrote a text layout engine for Python because I thought the alternatives sucked. I struggled with loading libraries to support vertically-oriented Japanese text and then it hit me "these characters are all squares... it can't be that hard!" Then there was the gradual epiphany that: (1) if you want to use serif typefaces and have it look good you have to kern them properly (2) Word, Powerpoint, and Adobe tools don't kern properly (3) that's why sans serif typefaces are so fashionable these days Of course I am using this to print cards with custom software and it doesn't have to be interactive, reflow, or be useful to anyone else.

froh 4y ago
Yes, and also ligatures, serif fonts love ligatures...
- PaulHoule 4y ago
  Another complaint I have is that if you look really close at most typefaces the letters and numbers seem to not just not come from the same typeface but not even from the same planet.
- dhosek 4y ago
  Which are (a) not unique to serifed faces (the original Avant-Garde famously was full of discretionary ligatures) and not necessarily ideal for serifed faces. Many typefaces designed for Linotype hot metal look decidedly worse with ligatures than without because the f could not have any overhang so if you had something like fifty, the two fs looked bizarre in conjunction. My go-to type for demonstrating this problem was Trump Mediaeval but sometime in the transition from Type 1 to Opentype, the type gnomes at Linotype saw the obvious issue here and replaced the ugly fi ligature with a logotype with fi without the ligature (the logotype existing more for the sake of documents which might have manually specified ﬁ rather than fi (I’m curious to see whether the difference is visible when this gets translated into its presentation form)).
dhosek 4y ago
Here’s the secret of typesetting vertical Japanese text: You turn all the characters anticlockwise 90°. It is now exactly the same process as typesetting any L-R text. Regarding your epiphany, (a) Word doesn't turn on kerning by default, but it does indeed kern properly. I don't use Powerpoint, but I'm guessing there might be a similar setting on the second tab of the font modal for turning on kerning. I don't know where you get your claim that Adobe tools don't kern properly, but they very much do. And finally, kerning is just as important for sans-serif type as it is for serifed type. The only arena it doesn't come into play is monospace type. Which, perhaps surprisingly, Japanese type traditionally is.

userbinator 4y ago

I'll offer the counterpoint that it's only complicated if you want to make it so. If you only need e.g. monospace ASCII, that doesn't take much. Do you really need proportional fonts? Vertical layouts? Keming? In my experience, a lot of the time the answer is actually "no" --- and thus the simpler (and more efficient, less bug-prone) algorithm works just fine.

zokier 4y ago
Do you really need more than 1s and 0s? Maybe go back to blinkenlights and toggle switches, those are simpler and even less bug prone to implement. You can still perform any computation you want, but it sure will not be as pleasant as having nice ux.
FooBarWidget 4y ago
Whether you need them is up to users. If you have Asian users then ASCII will not do. No kerning? Why would users put up with ugly text rendering? Just to make our lives as developers easier? This attitude reminds me of what someone says about the difference between German and Japanese tech. Japanese ones last forever because they overdesign their products to avoid breaking even when misused, because they are customer-focused. Germans are rule-focused, so they design their products according to spec. German products also last forever — if you use the product exactly per instructions. But if you make a mistake... good luck.
- userbinator 4y ago
  "ugly" is entirely subjective.
  - can16358p 4y ago
    Well, if 90 out of 100 users think it's ugly, that would better be resolved to have more users use your library, even if you are in the 10% as the developer.
- eska 4y ago
  As a German working in a Japanese company this kind of romanticization of these two cultures always makes me snort.

EdSchouten 4y ago

Parsing and displaying a floating point number, a la strtod() and printf(“%f”). I think that’s always the prime example of something that looks simple, but simply isn’t.

zokier 4y ago

Are there any simple software things that are actually very simple? I feel like tons of the very basic things that have been core software tasks for ages still have surprising amount of complexity and depth to them when you'd expect them to be simple. Stuff like text (basically anything), math (real numbers), 2d graphics (efficient high-quality vector graphics still seem like a pipedream), sorting (we just had a thread about improving sorting in postgres!), files (infamous fsync issues), color, etc just seem to constantly come up as problems when you'd think we had solved them already 20 years ago.

syntheweave 4y ago
Most software things become simple when you can define them down into an enumeration of possibilities. For example, if my text problem were "display these preformatted strings in a single resolution and layout" there would be no requirement on their encoding or processing and I could do whatever I like, even hardcode them or store them as image assets. Games have done this trick since forever because they have a tremendous capacity to soak up more and more static content, and much of the core tech in a game engine is in finding the ideal representations to efficiently load and present that content while carefully minimizing the dynamic behavior that would entail a general-purpose solution. The problem for software in the larger view of things is always that we've defined the problem with sufficient generality that it has to accommodate all the depth, because we don't know who is using the software or what they want from it. The medium is never "set in stone" so the tools are forever adapting to a new use case. And just when you think you've modelled every possible aspect of the data, some other way of doing it will come up. And it's not a right thing either to think "OK, I'll just solve the most general thing I can think of now so that I don't deal with it later" because then you just have a wide field of untested edge cases and no coverage of the thing you didn't anticipate. Like, take image editing software as an example. They all let you draw things freehand. Many support pressure sensitive stylus input. But the way they interpret that input is all over the map: different sensitivity curves, stabilization algorithms, brush behaviors and so on. There's no winning the battle by defining the most general engine for freehand drawing and painting, because what the user craves most of all here is the path to "just works" defaults. Thus in every sufficiently developed editor, an enumeration of possibilities appears again, but as a configuration letting you browse presets.

tester756 4y ago

http://johnsalvatier.org/blog/2017/reality-has-a-surprising-amount-of-detail http://johnsalvatier.org/blog/2017/reality-has-a-surprising-...

umvi 4y ago

Some of these things are only super hard if you are developing a general solution. The general solution to word wrapping across all written human languages in use on the planet is indeed difficult. But implementing basic word wrap for an HTML5 game that only English players will play isn't too hard.

bo0tzz 4y ago
Unless you're targeting a very localized audience (eg employees in one company, kids in one school district), I don't think it's safe to assume that only English users will use your software.
- Aeolun 4y ago
  It may not be safe to assume only English speakers will use your software, but it is certainly reasonable to support only them.
- adrianN 4y ago
  If your software only supports English, most of its users will be fine with English.

ivraatiems 4y ago

I agree with the general premise - sometimes "simple" things in software are really complex - but not the specific examples. The reason for the complexity in the author's examples is that they're using web canvases in a way they are not, as far as I know, intended to be used. Canvases are for 2D shapes/bitmap rendering, not complex text and user interactions. You're having trouble because you're reinventing the wheel. Maybe it's necessary for your use case, but that doesn't make it shocking that it's hard.

ec109685 4y ago
Unless you’ve done it before, there’s no way you would think it to be that complex to simulate the OS behavior. That was the point of the article. When reading that list, my thoughts turned to whether there is a good open source implementation that could be used / cross compiled into web assembly since surely somebody has done it correctly.

kccqzy 4y ago

Somewhat famously, Knuth came up with a dynamic programming algorithm for breaking paragraphs into lines with full justification (both left aligned and right aligned). Later on there are of course further innovations like character protrusion and expansion to achieve that optically aligned look without much (if any) hyphenation.

PaulDavisThe1st 4y ago
I am not sure it's clear that Knuth's work is appropriate for all or even any non-Roman alphabetic languages.
- svat 4y ago
  The best typeset books I've seen in Devanagari (a non-Roman script, used for Sanskrit, Hindi, Marathi, Nepali) have been typeset with XeTeX (XeLaTeX). Of course this involves other system libraries like Harfbuzz to handle the kerning and measure widths of individual words, and leaves some of the complexities mentioned in the post to the user, but the Knuth–Plass algorithm from TeX is still what is at the heart of it, for breaking paragraphs into individual lines. Take a look at TUGboat and the TUG conferences or the TeX.SE website to see it being used for many languages and scripts (including Arabic, Hebrew, etc). TeX is also big in Japan it appears, and they maintain their own variant engines (ptex, eptex, uptex, euptex).
  - PaulDavisThe1st 4y ago
    In the 80's, I did the typography design for two books with TeX and LaTeX, so it's not as if I'm unfamiliar with the stuff :) However, it is true that I haven't seen many examples of non-Roman script being well-set with TeX, so I guess I should take another look.
- cracrecry 4y ago
  It was used exactly for that application. Not that long ago, typesetting math formulas was extremely expensive, unless you used TeX. It was not very commercial and companies were not interested but certain individuals were. The same happened when you wanted your Hebrew or Arabic or Japanese just to look right. Japanese uses at least four scripts plus latin words in the same place. Over time, commercial entities added support for those, most of them just incorporating TeX source code (as it was public domain). They added the visual interfaces and the easiness of use that TeX lacks.

trhway 4y ago

I think the complexity we handle has stayed the same. It is just for the same amount of complexity and mental energy we get more functionality these days than decades ago. Text is a good example - the same complexity and the same amount of code handles these days a huge number of languages whereis decades ago that complexity would give you only one.

nraynaud 4y ago

I had a client simultaneously telling me he wanted a function, but that it couldn’t take that much time to develop it because it’s only useful 5% of the time. Sorry dude it’s all or nothing, it’s not like mechanics were there is a range of solutions going from sheet metal to forged titanium or from a piano hinge to magnetic bearings. I’m a bit unfair, because some high tech stuff is cheap, we all use tree and hash structures whose development we don’t pay because they are in the standard library, but it’s a few strategic and standardized places. The same as using a ball bearing in mechanics: you get into a very high tolerance world for cheap, the mechanical interface is standard and some unicorn mounted helve make those bearing in a chocolate factory as far as I know they might develop maps and queues on their other half time.

Gigachad 4y ago

A lot of the complexity in this article isn't so much that the task is very complex, but that if you want to support every single language, culture, format, then it becomes a very long task. I do wonder if we will end up just moving towards more unified formats and doing away with the need for all this complexity. I could be very wrong on this, but it seems like for the smaller to mid scale software projects, I18n is just not worth it. Throwing a bunch of YAML files over the wall to a translation company and then just dropping them in the app does not work and the end result is so bad that native speakers would rather just use the app in correct English than broken native language. Now you don't have to care about how to wrap or hyphenate a word in an obscure language which doesn't support it, and you don't have to worry about vertical text. Your problem space just got a lot smaller. And if you are building a tool at Google scale then you should have the resources to just make the problem go away anyway.

colordrops 4y ago
Supporting "every single language, culture, format" *is" the task in the article. You can decide whether you want to do it or not, but that's a different question.
- AnimalMuppet 4y ago
  But who thinks that's simple? That's an insanely complex task.
  - Gigachad 4y ago
    At a high level the task may seem simple "Wrap the text when it reaches the end of the line". And if you only care about English, it actually is reasonably simple. The complexity is stuff you wouldn't even think about unless you had experience in the area.
    - AnimalMuppet 4y ago
      Fair, to a point. But colordrops said "supporting every single language, culture, format". So by definition, it's more than English. And as soon as you start looking into non-English writing systems, you realize that it is very much not simple.

PaulDavisThe1st 4y ago

I found it amusing to read this, given that in the context of Ardour (0) we still have the long term stretch goal of replacing GTK with an expansion of our home-grown canvas-based widget system. There are 5 widgets missing, one of them is the text entry. The description in TFA regarding doing this in the context of an HTML canvas element is more less identical to the issues that exist for a native re-implementation, and there's consequently a similar high pressure rationale for giving up on the idea. (0) https://ardour.org/ https://ardour.org/

robocat 4y ago
https://workspaceupdates.googleblog.com/2021/05/Google-Docs-Canvas-Based-Rendering-Update.html https://workspaceupdates.googleblog.com/2021/05/Google-Docs-... mentions some of the other complications like assistive screen-readers, extensions, and how add-ons work without HTML. https://docs.google.com/document/d/1vUnn0ePU-ynArE1OdxyEHXR2G0sl74ja_st_4OOzlgE/preview https://docs.google.com/document/d/1vUnn0ePU-ynArE1OdxyEHXR2... feels very weird on an iPad from a quick 10 second look: selection markers are Android styled, scrolling is janky, scroll easing is wrong, no scrollbar. Popups menus are very different and they have Mac keyboard shortcuts! Etcetera etcetera, and that was on a readonly document. (I presume it is the current version of Google Docs from a browser).

ronnier 4y ago

I've come to the conclusion that nothing is simple. Nothing. It's only "simple" because thousands of years of work were applied to get us to where we are today and we are building on top of that. It's arrogant to think anything is simple. Just to build a key cap for this laptop I'm typing on, very difficult if I actually had to do that. I need to use tools that were ultimately kicked off by stone age men thousands of years ago that progressed by building tools with those tools until we go to the point today where we can create a key cap for a laptop.

mrwh 4y ago
I'm sure you know this story already, but nice example: https://gizmodo.com/one-mans-nearly-impossible-quest-to-make-a-toaster-from-5794368 https://gizmodo.com/one-mans-nearly-impossible-quest-to-make...
- ghgr 4y ago
  Thanks for sharing! It brought back to my mind this project of mine (more like a "wish" than an actual project) to build the current technology tree (like Age of Empire's or Factorio's) as a sort of wiki. And of course the wonderful essay by Leonard E. Read "I, Pencil".
- amelius 4y ago
  A better example would be someone sitting in the middle of a jungle with no access to technology trying to make a toaster from scratch.
- deleted 4y ago
  [deleted]
dwohnitmok 4y ago
Oh it goes even further than that, all the way to the Big Bang. As Carl Sagan said, “If you wish to make an apple pie from scratch, you must first invent the universe.”
meribold 4y ago
> I've come to the conclusion that nothing is simple. Surely it's more complicated than that ;)

jimbob45 4y ago

Browser text isn’t measurable AFAIK at least in any consistent way. I tried doing it for a project at work and came to the conclusion that it simply wasn’t possible. The SO answer below appears to agree with me. https://stackoverflow.com/a/12469203 https://stackoverflow.com/a/12469203

joshvm 4y ago

There are plenty of variations on this, I'm surprised no one has mentioned Falsehoods programmers believe about names yet: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... (as someone with a long, double barreled surname this is relatable) The common thread is usually what assumptions you can make about your use case, when are they valid and when do they fail?

raphlinus 4y ago
I think of "text rendering hates you"[1] as the classic "falsehoods" analog about text. [1]: https://gankra.github.io/blah/text-hates-you/ https://gankra.github.io/blah/text-hates-you/

buescher 4y ago

A whole lot of the simple functions in the standard C library are in there because a naive implementation will have bugs, often subtle ones. Implementing strcpy or strlen used to be a common interview question.

ozzythecat 4y ago

There are two things I absolutely do not want to implement, ever, if I can get away with it. Any kind of dynamic page rendering: we want to write code once and re-use everywhere, in the Web browser, mobile, iOS, Android, etc. This is a fool’s errand 99% of the time. You’ll end up building something super constrained if it is usable at all. Admittedly there are some viable options for this like React Native. Yeah, I just don’t want to work on this type of code. Client libraries: I want to write my own client, in top of a third party networking library, to send HTTP requests, and handle threading, tcp connections, connection pools, thread pools, etc. Even when using something as simple as libwebcurl, innocuous things fail in weird, unexpected ways that almost never come up in (my) unit tests, integration tests, and chaos tests. Server side client code that calls another service can be dangerous. Device code that you ship out to millions of devices is even more dangerous. Oh… 50% error rate because suddenly the code can’t access an SSL cert on the device. Wait… what??? I can’t even reproduce it? I’ve been writing code since grade school and am a decade now into my career. I’m an above average programmer on a good day, and terribly average at best on most days. So of course, I might be sounding melodramatic here.

gernb 4y ago

Now if flutter would learn all of that and stop trying (and failing) reproduce it! Guess what? OS Level and browser level spell checking doesn't work in Flutter because it's rendering pixels instead of OS text input / HTML text input. And a ton of other issues that will likely never be fixed. It almost makes we wonder if Flutter's goal isn't good UX but rather user control. If it's just pixels it's much harder for users to copy and paste, quote, scrape, etc....

ec109685 4y ago
Interesting they don’t use hidden native elements to simulate that. I guess kerning and layout would be subtly different between platforms. I wonder what Google Docs does on iOS? ReactNative doesn’t have this problem since the ui is expressed ultimately as native components.

dhosek 4y ago

Grapheme segmentation in Unicode is actually kind of interesting and completely data-driven. I wrote my own implementation of the segmentation algorithm in Rust recently¹ (because the existing segmentation crate didn’t have an interface that let me get the next grapheme cluster from a char iterator). It was fascinating to dig into the specifications and see how everything fits together. ⸻ 1. It will be published to crates.io sometime in the next month or so, there’s possible optimization work to be done, or at least, an alternate interface to the iterator to see whether the speed penalty I’m paying is an algorithmic problem or just an artifact of the trade-offs I have for my use case. I also ended up writing my own character class code as well and in benchmarking against the current popular crate got a 10x speedup,² so even if my segmentation code is useless to anyone but myself, I think that this code at least will be useful. 2. The other crate has a really inefficient implementation of looking up character classes where it ends up having to do a binary search in the data tables for many characters while I use a two-step table approach which lets me get a character’s class in O(1) time (and the tables are designed to be small enough that they’ll stick around in the CPU cache for lookups in a tight loop).

kwantam 4y ago
Sounds like a great improvement! Would it be possible to contribute your code to the existing crate rather than release your own? Then we could all enjoy the benefits on the next recompile :)
- dhosek 4y ago
  The existing crate seems to have lost its maintainer.

deathanatos 4y ago

No, text on a canvas is a somewhat obvious nightmare. No, we must go simpler. left-pad: you need to account for wide characters, such as CJK characters but also emoji, you need to account for things like combining characters, etc. left-pad is not some trivial function. Of course, the infamous left-pad package was widely mocked "why do you need a whole package for such a simple task" — so simple, indeed, that it failed to handle many of those cases. But don't worry, it's been standardized now as padStart() … and still fails those cases: > 'e\u0301'.padStart(2) 'é' Okay, strings aren't simple! But numbers, numbers are fine right? So then, Go issue #4594, "add a Round function [to Go]"[1]. Go maintainer Ross Cox suggests that "it's a one liner" and suggests … a buggy implementation. There are a total of no less than 6 different implementations in the issue, and every single one of them is buggy. The bug is closed, the function is "too simple" to add. Thankfully, the wound gets reopened nearly a half-decade later in #20100. Another buggy implementation is suggested in that thread, and thankfully, Go gets a round function. (A non-buggy one … I think.) [1]: https://github.com/golang/go/issues/4594 https://github.com/golang/go/issues/4594

mro_name 4y ago
the basics are incredibly hard. Listen to a physicist explaining time, matter or energy.
- deleted 4y ago
  [deleted]
gernb 4y ago
The entire idea of padding by characters only fits a few use cases. leftPad would never have covered those cases. If you want tabular data use a table or grid. No amount of string gymnastics will get you aligned strings across languages and fonts. So yea, leftPad was always a bad idea because the only useful implementation is too trivial too have been put in a 3rd party library. Unless you believe there should have been a library called 'add' that was just `export const add = (a, b) => a + b;`

orangepurple 4y ago

I was going to nominate mutexes and semaphores

lelanthran 4y ago

I don't know of any developer that thinks text rendering is "simple". Even some non-devs, due to using word-processors that completely "mess-up" everything the minute a single table is added, won't think that text rendering is simple. The best "simple-is-very-complicated" essay to read is "Reality has a surprising amount of detail" - http://johnsalvatier.org/blog/2017/reality-has-a-surprising-amount-of-detail http://johnsalvatier.org/blog/2017/reality-has-a-surprising-...

ec109685 4y ago
That was an excellent blogpost. This whole discussion reminds me of Joel’s Single Biggest mistake software developers do: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-... Namely, rewriting software is hard because the years of subtle complexity aren’t immediately apparent and hard to get right the second time (at least harder than first thought).

magic_hamster 4y ago

You got to leave out implementing semantic considerations when starting out. I.e. if your line break changes the meaning of a word, or using hyphens is not allowed in certain words for some languages, those are not technical challenges. They are semantic challenges and solving them will require heuristics or specialized handling. Although being aware of these considerations is helpful for designing a solution which will be able to address these issues later. Like determining allowed break spots before doing the actual line break. And having that logic extensible.

ycuser2 4y ago

Unrelated to the content: What is the reason to link to external websites like this: https://www.construct.net/out?u=https%3a%2f%2fen.wikipedia.org%2fwiki%2fInput_method https://www.construct.net/out?u=https%3a%2f%2fen.wikipedia.o... Why not set a link to wikipedia directly?

leksak 4y ago
To track if people clicked on the link maybe?

bribri 4y ago

There are many hidden complexities in seemingly simple features like text input fields and wrapping text in a canvas. It is better to let the browser or operating system handle these features, as they have been perfected over many years and are much more likely to work correctly than a custom implementation.

bullen 4y ago

I'm doing .ttf OpenGL with custom row break for my MMO: The trick is to take as much advantage of this command-line and remove all other GUI. I'm only going to support ASCII which helps a great deal. Chat bubble / command-line above characters is the only GUI the game will have: Login: /sign name pass Register: /join name pass pass Sensitivity: /sens 0.6 etc.

dhosek 4y ago

>Splitting strings to graphemes is actually quite slow, so you can't do it too often during word wrap, or you'll tank performance. If you're splitting strings to graphemes for word wrap, you're doing it wrong. Unicode segmentation defines a word segmentation algorithm that should be used for that use case instead.