4 ms·

I think there's lots of scope for improvements to terminals, but I feel like this is more a question of "nobody has asked for it". There's been plenty of recen

by vidarh 1y ago

I think there's lots of scope for improvements to terminals, but I feel like this is more a question of "nobody has asked for it". There's been plenty of recent innovation in terminals (e.g. support for a variety of new types of underlines to enable "squigglies" for error reporting is an example; new image support is another), and adding a code to enable more detailed key reporting the same way we have upgraded mouse event reporting over the years wouldn't be hard, and these things tends to spread quickly. With respect to "accidentally printing a line that's too long", you can turn off auto-wrap in any terminal that supports DECAWM (\033[?7h / \033[?7l ). That it's "kinda slow" really shouldn't be an issue - it was fast enough for hardware a magnitude slower than today. Parsing it requires a fairly simple state machine. If can't keep up with VT100/ANSI escape sequences, your parser is doing something very wrong. The difficulty of unicode is fair enough, and sadly largely unavoidable, but that part is even worse in a GUI; the solution there is to use code to measure the rendered string, and it's not much harder to get that right for terminals either. It'd be nice if unicode had done this in a nicer way (e.g. indicated it in the encoding). For my own terminal, I'm toying with the idea of allowing proportional text with an escape code, and make use of it in my editor. If I do, it'll be strictly limited: Indicate a start and end column where the text is proportional, and leave it to the application to specify a font and figure out the width itself. Worst case scenario would be that you send the escape, and the editor doesn't get an escape acknowledging it has been enabled back, and falls back on monospaced text and keeps working fine in a regular terminal. This way, evolving terminal capabilities can be done fairly easily with backwards compatibility.

akkartik 1y ago
Anything is possible to fix; the question is why bother. Every fix cuts into the benefit of compatibility. The fundamental model of a wrapping/scrolling teletype isn't a good fit for the way we use computers today. (It does make sense if you work in a real text mode console. Then you are really avoiding all the complexity of a graphics stack by using hard-coded capabilities your hardware provides.) A simple flat array of pixels seems like a much more timeless mental model to build durable software on top of. You don't have to wonder how different computers will react to a write just off the bottom right of the screen, and so on.
- vidarh 1y ago
  The reason to bother is that a lot of prefer terminals and want to evolve them. The reason they're not evolving faster isn't that compatibility is really a problem because we see new terminal capabilities gain support fairly quickly, but usually because there often isn't a major perceived need for the features people who don't use terminals much think are missing. People don't add capabilities to try to attract people like you who don't want terminals in the first place. Wrapping and scrolling can be turned off on any terminal newer than the vt100, or constrained to regions. I never wonder how a different computer reacts to writing off the bottom right of the screen, because that works just fine on every terminal that matters. The actual differences are relatively minor if you don't do anything esoteric. A "simple" flat array of pixels means you have to reimplement much of a terminal, such as attribute rendering, font rendering etc. It's not a huge amount of work, but not having to is nice. So is the network transparency, and vnc etc. is not a viable replacement.
  - akkartik 1y ago
    Availability over ssh is indeed a good point. I've reduced my reliance over the network at the same time I've grown disenchanted with terminals; thanks for pointing out that connection. The rest are mutually incommensurable worldviews, and we have to agree to disagree.
  - akkartik 1y ago
    "A simple flat array of pixels means you have to reimplement much of a terminal, such as attribute rendering, font rendering etc. It's not a huge amount of work, but not having to is nice." For me the debate isn't about implementing a terminal vs something else. I assume one uses tools others build in either case. The question is how much the tools hinder your use case. I find a canvas or a graphical game engine (which implement fonts and so on) hinders me less in building the sorts of tools I care about building. A terminal feels like more of a hindrance.
    - vidarh 1y ago
      > I assume one uses tools others build in either case And there is the disconnect. For a terminal app, you often don't need to. > I find a canvas or a graphical game engine (which implement fonts and so on) hinders me less in building the sorts of tools I care about building. A terminal feels like more of a hindrance. And for me it's the opposite. The tools I build mostly works on text. A terminal provides enough that I usually don't need any extra dependencies.
kragen 1y ago
I think I've said this to you before, but in case I haven't, I suspect that the best way to handle proportional text in formattable plain text is some variant of Nick Gravgaard's "elastic tabstops" https://nickgravgaard.com/elastic-tabstops/ https://nickgravgaard.com/elastic-tabstops/: > Each cell ends with a tab character. A column block is a run of uninterrupted vertically adjacent cells. A column block is as wide as the widest piece of text in the cells it contains or a minimum width (plus padding). Text outside column blocks is ignored [for layout, though it is displayed]. I think the main deficiency in his proposal is that he enforces a minimum of one space of whitespace padding on the right of his column.
- vidarh 1y ago
  I love that. Thanks. I think you can avoid making that padding visible. In terms of making this work in a terminal, what I'd imagine would be having the app still aligning things to fixed-width column boundaries, but handling the elastic tab stops based on knowing a maximum extent of a single-width character used (you'd still need to deal with the unicode mess) and setting tab stops based on that. Adding an escape to report the font extents would be easy. I'll have to do some experiments on this... Especially as I'm the kind of maniac who likes to align multi-line spans of code in way linters will yell at.
  - kragen 1y ago
    I thought you might like it! I was thinking you could change the terminal to implement elastic tabstops so that apps written for the elastic-tabstop terminal wouldn't have to worry about font metrics. You could use ^L and ^V as delimiters to nest a whole new table inside a table cell, giving you layout capabilities comparable to the WWW before CSS, similar to how HN is laid out. Languages like C and JS that treat all whitespace the same can have ^L and ^V inserted freely in random places.
    - vidarh 1y ago
      I'm going to have to think about exactly how to do this best. My one concern would be to either handle line-wrap properly, or handle overflow. Overflow is "easy" if you assume a line is always of a reasonable max length: Just turn off wraparound, and print the whole line. But if you want to handle super-long lines, you'd at least want a way to "clip" the text you're sending to the terminal to avoid the cost of outputting all of it to the terminal each time you're moving past it. Depending on how, maybe the minimum width of a glyph is instead what you worry about in cases where you don't wrap. Let's say the terminal is 80 monospaced characters wide, and you switch on proportional mode. If you have a long line, you just want to know the maximum number of characters you should output to guarantee that the proportional region is full. Maybe just an escape to get the terminal to report the maximum number of proportional glyphs that will fit in a given field is enough. The worst case scenario then is a screen full of I's or 1's or similar, where maybe you spit out twice as much text that the terminal propmptly clips away, but most of the time you'd only output a few more characters than necessary, so I think that's fine. Not sure how to cleanly extend that to allow wraparound without the app needing to be aware of the font. E.g. a text editor will want to be able to figure out which point in the text should be at the top of the screen as you scroll. Doing that with wraparound without knowing font extent would require some other way to measure. Printing a bit extra might be fine. For non-interactive applications, none of this would matter - you'd just print everything the same way you do now, and let the terminal sort out the rest. Maybe adding experimental support for that would be a good starting point. E.g. being able to cat a tab-separated file and get an aligned table with proportional text would be an awesome test case. I'm now also curious how much would break if I "just" start doing proportional rendering by default when not using the alternate screen. E.g. anything trying to do layout with spaces would obviously break, but I'm wondering if maybe it'd affect few enough non-interactive programs that making that the default, and just wrap/alias the few things that'd break. My shell already filters output of a number of commands to colorize them, like pstree, so having a small "blacklist" of programs I use that'd need to be switched back to monospaced, or filtered further, wouldn't necessary be a big deal. Damn you for adding more things to my TODO list :-)

by vidarh 1y ago

akkartik 1y ago

Anything is possible to fix; the question is why bother. Every fix cuts into the benefit of compatibility. The fundamental model of a wrapping/scrolling teletype isn't a good fit for the way we use computers today. (It does make sense if you work in a real text mode console. Then you are really avoiding all the complexity of a graphics stack by using hard-coded capabilities your hardware provides.) A simple flat array of pixels seems like a much more timeless mental model to build durable software on top of. You don't have to wonder how different computers will react to a write just off the bottom right of the screen, and so on.

vidarh 1y ago
The reason to bother is that a lot of prefer terminals and want to evolve them. The reason they're not evolving faster isn't that compatibility is really a problem because we see new terminal capabilities gain support fairly quickly, but usually because there often isn't a major perceived need for the features people who don't use terminals much think are missing. People don't add capabilities to try to attract people like you who don't want terminals in the first place. Wrapping and scrolling can be turned off on any terminal newer than the vt100, or constrained to regions. I never wonder how a different computer reacts to writing off the bottom right of the screen, because that works just fine on every terminal that matters. The actual differences are relatively minor if you don't do anything esoteric. A "simple" flat array of pixels means you have to reimplement much of a terminal, such as attribute rendering, font rendering etc. It's not a huge amount of work, but not having to is nice. So is the network transparency, and vnc etc. is not a viable replacement.
- akkartik 1y ago
  Availability over ssh is indeed a good point. I've reduced my reliance over the network at the same time I've grown disenchanted with terminals; thanks for pointing out that connection. The rest are mutually incommensurable worldviews, and we have to agree to disagree.
- akkartik 1y ago
  "A simple flat array of pixels means you have to reimplement much of a terminal, such as attribute rendering, font rendering etc. It's not a huge amount of work, but not having to is nice." For me the debate isn't about implementing a terminal vs something else. I assume one uses tools others build in either case. The question is how much the tools hinder your use case. I find a canvas or a graphical game engine (which implement fonts and so on) hinders me less in building the sorts of tools I care about building. A terminal feels like more of a hindrance.
  - vidarh 1y ago
    > I assume one uses tools others build in either case And there is the disconnect. For a terminal app, you often don't need to. > I find a canvas or a graphical game engine (which implement fonts and so on) hinders me less in building the sorts of tools I care about building. A terminal feels like more of a hindrance. And for me it's the opposite. The tools I build mostly works on text. A terminal provides enough that I usually don't need any extra dependencies.

kragen 1y ago

I think I've said this to you before, but in case I haven't, I suspect that the best way to handle proportional text in formattable plain text is some variant of Nick Gravgaard's "elastic tabstops" https://nickgravgaard.com/elastic-tabstops/ https://nickgravgaard.com/elastic-tabstops/: > Each cell ends with a tab character. A column block is a run of uninterrupted vertically adjacent cells. A column block is as wide as the widest piece of text in the cells it contains or a minimum width (plus padding). Text outside column blocks is ignored [for layout, though it is displayed]. I think the main deficiency in his proposal is that he enforces a minimum of one space of whitespace padding on the right of his column.

vidarh 1y ago
I love that. Thanks. I think you can avoid making that padding visible. In terms of making this work in a terminal, what I'd imagine would be having the app still aligning things to fixed-width column boundaries, but handling the elastic tab stops based on knowing a maximum extent of a single-width character used (you'd still need to deal with the unicode mess) and setting tab stops based on that. Adding an escape to report the font extents would be easy. I'll have to do some experiments on this... Especially as I'm the kind of maniac who likes to align multi-line spans of code in way linters will yell at.
- kragen 1y ago
  I thought you might like it! I was thinking you could change the terminal to implement elastic tabstops so that apps written for the elastic-tabstop terminal wouldn't have to worry about font metrics. You could use ^L and ^V as delimiters to nest a whole new table inside a table cell, giving you layout capabilities comparable to the WWW before CSS, similar to how HN is laid out. Languages like C and JS that treat all whitespace the same can have ^L and ^V inserted freely in random places.
  - vidarh 1y ago
    I'm going to have to think about exactly how to do this best. My one concern would be to either handle line-wrap properly, or handle overflow. Overflow is "easy" if you assume a line is always of a reasonable max length: Just turn off wraparound, and print the whole line. But if you want to handle super-long lines, you'd at least want a way to "clip" the text you're sending to the terminal to avoid the cost of outputting all of it to the terminal each time you're moving past it. Depending on how, maybe the minimum width of a glyph is instead what you worry about in cases where you don't wrap. Let's say the terminal is 80 monospaced characters wide, and you switch on proportional mode. If you have a long line, you just want to know the maximum number of characters you should output to guarantee that the proportional region is full. Maybe just an escape to get the terminal to report the maximum number of proportional glyphs that will fit in a given field is enough. The worst case scenario then is a screen full of I's or 1's or similar, where maybe you spit out twice as much text that the terminal propmptly clips away, but most of the time you'd only output a few more characters than necessary, so I think that's fine. Not sure how to cleanly extend that to allow wraparound without the app needing to be aware of the font. E.g. a text editor will want to be able to figure out which point in the text should be at the top of the screen as you scroll. Doing that with wraparound without knowing font extent would require some other way to measure. Printing a bit extra might be fine. For non-interactive applications, none of this would matter - you'd just print everything the same way you do now, and let the terminal sort out the rest. Maybe adding experimental support for that would be a good starting point. E.g. being able to cat a tab-separated file and get an aligned table with proportional text would be an awesome test case. I'm now also curious how much would break if I "just" start doing proportional rendering by default when not using the alternate screen. E.g. anything trying to do layout with spaces would obviously break, but I'm wondering if maybe it'd affect few enough non-interactive programs that making that the default, and just wrap/alias the few things that'd break. My shell already filters output of a number of commands to colorize them, like pstree, so having a small "blacklist" of programs I use that'd need to be switched back to monospaced, or filtered further, wouldn't necessary be a big deal. Damn you for adding more things to my TODO list :-)