5 ms·

In theory, `reset`/`tset` from ncurses, or an equivalent, is this "thing" that programs are supposed to be using when they need to engage in terminal shenanigan

In theory, `reset`/`tset` from ncurses, or an equivalent, is this "thing" that programs are supposed to be using when they need to engage in terminal shenanigans. Those use TERM/TERMCAP environment variables and some educated guessing reading stdin/stdout/stderr behaviour to figure out what terminal you're using, search for said entry in terminfo/termcap database, and finally invoking native terminal behavior or simulating it with other terminal behavior. In theory, one could "just implement" an alternative terminal, and put said simplified entries in terminfo/termcap database and things will be all fine. In practice several of those ANSI escape code behaviors are hard-coded and everyone pretends everything is "at least as capable as xterm" (a oxymoron, given that xterm is one of the most capable and feature-rich terminals around), so you're also gonna have to implement a converter from "xterm ANSI" to your alternative terminal system - sort of how winpty does convert Win32 terminals conventions to ANSI ones. Then we're back to the starting issue in this converter.

andrewla 3y ago
Yes, this is one of the avenues that I've chased down a bit. One problem is termcap/terminfo itself. The definition has expanded greatly over time, and documentation is poor to non-existent. In many cases termcap/terminfo has been implicitly extended to support ncurses specifically. And like you say, everyone just sort of assumes ANSI/xterm -- nobody has `PS1="$(tput setaf 3)\w $(tput setaf 0) $", you just hard-code the codes. The last time I chased the rabbit down this hole I got mired in TTY-land and was not able to dig myself out. Too many insane layers, and before you know it you're writing code to emulate serial connections in a vain attempt to make things work without burning the whole world down. Both (n)vim and tmux have internal TTY emulation as well which makes things even crazier; I think vim uses libvterm and tmux has one internally hand-coded. It's a mess.
- db48x 3y ago
  tmux, screen, emacs, and vim all have terminal emulators in them because it’s required, not because they’re crazy. Consider a really simple case. You are using screen (or tmux, or emacs, or whatever) and you have your terminal divided in half so that you can use a program in one half and tail a log file in the other. The program sends a ^L to clear the screen. Screen (or tmux or whatever) must read the ^L and then _not_ send it on. If it sent it on, the whole screen would be cleared, not just the half with the program in it. Instead it has to send whatever control sequences would erase the text from just the right portion of the screen while leaving everything else alone. Virtually everything sent by the program(s) has to be intercepted and reinterpreted in order for it to work right.
  - kps 3y ago
    No, they're crazy. I already have a window manager I like; I don't need my terminal to implement their own half-assed one, and my multiplexer (running on the terminal) to implement their own half-assed one, and my editor (running on the multiplexer on the terminal) to implement their own half-assed one.
    - hnlmorg 3y ago
      So don't use tmux then. The entire reason most people use something like tmux is because they want a window manager in their terminal. This is one of the great joys of FOSS, if you don't like a terminal-based utility then you can bet there's a dozen other options available for you to use.
    - db48x 3y ago
      If you really want crazy, run `xterm -ti 340`, then run run an X server from the xserver-sixel repository <https://github.com/saitoha/xserver-SIXEL https://github.com/saitoha/xserver-SIXEL> in it. Now y ou can run as many terminal emulators, complete with real truetype fonts and all the colors you could want, inside the one terminal. Use a tiling window manager and you’ll be able to avoid using tmux entirely.
    - matvore 3y ago
      > No, they're crazy. I already have a window manager I like; I don't need my > terminal to implement their own half-assed one, Basically agree, though note that ssh and dtach and similar tools need to create pty's on the server and client ends because they implement essentially an adapter between a tty and a byte stream.
  - hnlmorg 3y ago
    I agree with your overall point but your example isn't a great one because ^L would be handled by the controlling process regardless of the terminal state. You don't need any special magic here. And blanking specific sections of a screen wouldn't need any special magic unless you're launching other processes and forwarding their output to the terminal. But if you're not forwarding the output of other processes then you can use standard ANSI escape sequences (or even just \s) to blank specific parts of the screen. \s is a bit hacky for blanking though, it can produce some undesired results in terms of copy/pasting from the terminal and resizing. But if you've got a "full screen" process like you're describing, you'd be trapping SIGWINCH for resize events anyway.
- hnlmorg 3y ago
  Terminal multiplexers like tmux need to create a pseudo-TTYs otherwise they couldn’t multiplex your terminal. My shell also creates PTYs on some rare occasions too. I’d prefer not to because it’s messy but sometimes there’s no way around it if you want your command line applications to behave like they would on a TTY whilst still having your utility capture that applications output as if it were writing to a pipe. This is a limitation of TTYs. The most annoying part of it all is that creating a PTY differs from one Unix-like OS to another. So it’s definitely not a rabbit hole developers go down happily.
  - kps 3y ago
    > Terminal multiplexers like tmux need to create a pseudo-TTYs otherwise they couldn’t multiplex your terminal. [Edit: I misread the parent comment as saying multiplexers needed to do terminal emulation. Dtach only multiplexes (using pesudo-TTYs, of course) but unlike screen or tmux does not do terminal emulation. Original comment follows:] That isn't strictly true; the one I use (dtach) is just a multiplexer.
    - hnlmorg 3y ago
      dtach creates pseudo-TTYs too: - https://github.com/crigler/dtach/blob/master/master.c#L53 https://github.com/crigler/dtach/blob/master/master.c#L53 - https://github.com/crigler/dtach/blob/master/master.c#L108 https://github.com/crigler/dtach/blob/master/master.c#L108 As an aside, that projects source is surprisingly easy to follow. I was able to grok it in just a couple of minutes. Kudos to the author.
      - kps 3y ago
        Yes, I misread your comment, sorry, having had my mind stuck on terminal emulation.
        hnlmorg 3y ago
        A PTY is an intrinsic part of terminal emulation. I think what you’re alluding to is that tmux gobbles up escape sequences (which is configurable by the way) and tracks cell positions. But as I said in another comment, the value add of tmux is its window management capabilities. That’s precisely why people like myself use it. And you cannot have that without complexity. In some edge cases that complexity causes issues, but on the whole I’m more productive for it. Saying it’s stupid that tmux does that is like saying it’s stupid that motorbikes have gears and fast patrol engines when a bmx works just as well. Sure, to someone who’s never seen a bike before they both look similar. But don’t be fooled by the fact that they both have 2 wheels because they cater to very different needs, hence their very different engineering designs.
foresto 3y ago
> In practice several of those ANSI escape code behaviors are hard-coded The most common examples I see these days are for red/green/yellow status text in shell scripts. To anyone planning to do that, or even more complex stuff, please consider using tput instead. tput setaf 1; echo this is red tput setaf 2; echo this is green tput setaf 3; echo this is yellow tput setaf 8; echo this is dim tput bold; echo this is bold tput sgr0; echo this is normal blue=$(tput setaf 4); normal=$(tput sgr0); echo ${blue}Color${normal} text. (man terminfo for arcane details.) It's wise to make sure output is going to a terminal before using colors, because they don't play well with logs or text processing commands. You can check with: test -t 1 If you use colors, please consider making them optional, since they don't work well for everyone or in every desktop color scheme. An environment variable makes a sensible switch.
- kps 3y ago
  > The most common examples I see these days are for red/yellow/green status text in shell scripts. Please don't do that without first checking the background colour. Yellow in particular on a light background is unreadable. (On an xterm-compatible terminal you can get the current background colour with OSC 11 ; ? BEL. Some popular terminals lie about being xterm-compatible, though.)
- LinuxBender 3y ago
  Make sure one has ncurses installed before switching to tput. Alpine Linux does not install this by default for what it's worth.
  - foresto 3y ago
    # No tput, no cry: red=$(tput setaf 1 2>/dev/null || true) echo ${red}text
    - LinuxBender 3y ago
      You tryin' to get Bob Marley stuck in my head? Because it worked.
  - 3y ago
kps 3y ago
Several popular terminals default to calling themselves xterm (e.g. `TERM=xterm-256color`), but are not xterm-compatible, and ignore bug reports (*cough* GNOME *cough*).
- audidude 3y ago
  Interesting. I don't work on VTE but just landed dozens of patches with no issue at all from pleasant maintainers.

andrewla 3y ago

Yes, this is one of the avenues that I've chased down a bit. One problem is termcap/terminfo itself. The definition has expanded greatly over time, and documentation is poor to non-existent. In many cases termcap/terminfo has been implicitly extended to support ncurses specifically. And like you say, everyone just sort of assumes ANSI/xterm -- nobody has `PS1="$(tput setaf 3)\w $(tput setaf 0) $", you just hard-code the codes. The last time I chased the rabbit down this hole I got mired in TTY-land and was not able to dig myself out. Too many insane layers, and before you know it you're writing code to emulate serial connections in a vain attempt to make things work without burning the whole world down. Both (n)vim and tmux have internal TTY emulation as well which makes things even crazier; I think vim uses libvterm and tmux has one internally hand-coded. It's a mess.

db48x 3y ago
tmux, screen, emacs, and vim all have terminal emulators in them because it’s required, not because they’re crazy. Consider a really simple case. You are using screen (or tmux, or emacs, or whatever) and you have your terminal divided in half so that you can use a program in one half and tail a log file in the other. The program sends a ^L to clear the screen. Screen (or tmux or whatever) must read the ^L and then _not_ send it on. If it sent it on, the whole screen would be cleared, not just the half with the program in it. Instead it has to send whatever control sequences would erase the text from just the right portion of the screen while leaving everything else alone. Virtually everything sent by the program(s) has to be intercepted and reinterpreted in order for it to work right.
- kps 3y ago
  No, they're crazy. I already have a window manager I like; I don't need my terminal to implement their own half-assed one, and my multiplexer (running on the terminal) to implement their own half-assed one, and my editor (running on the multiplexer on the terminal) to implement their own half-assed one.
  - hnlmorg 3y ago
    So don't use tmux then. The entire reason most people use something like tmux is because they want a window manager in their terminal. This is one of the great joys of FOSS, if you don't like a terminal-based utility then you can bet there's a dozen other options available for you to use.
  - db48x 3y ago
    If you really want crazy, run `xterm -ti 340`, then run run an X server from the xserver-sixel repository <https://github.com/saitoha/xserver-SIXEL https://github.com/saitoha/xserver-SIXEL> in it. Now y ou can run as many terminal emulators, complete with real truetype fonts and all the colors you could want, inside the one terminal. Use a tiling window manager and you’ll be able to avoid using tmux entirely.
  - matvore 3y ago
    > No, they're crazy. I already have a window manager I like; I don't need my > terminal to implement their own half-assed one, Basically agree, though note that ssh and dtach and similar tools need to create pty's on the server and client ends because they implement essentially an adapter between a tty and a byte stream.
- hnlmorg 3y ago
  I agree with your overall point but your example isn't a great one because ^L would be handled by the controlling process regardless of the terminal state. You don't need any special magic here. And blanking specific sections of a screen wouldn't need any special magic unless you're launching other processes and forwarding their output to the terminal. But if you're not forwarding the output of other processes then you can use standard ANSI escape sequences (or even just \s) to blank specific parts of the screen. \s is a bit hacky for blanking though, it can produce some undesired results in terms of copy/pasting from the terminal and resizing. But if you've got a "full screen" process like you're describing, you'd be trapping SIGWINCH for resize events anyway.
hnlmorg 3y ago
Terminal multiplexers like tmux need to create a pseudo-TTYs otherwise they couldn’t multiplex your terminal. My shell also creates PTYs on some rare occasions too. I’d prefer not to because it’s messy but sometimes there’s no way around it if you want your command line applications to behave like they would on a TTY whilst still having your utility capture that applications output as if it were writing to a pipe. This is a limitation of TTYs. The most annoying part of it all is that creating a PTY differs from one Unix-like OS to another. So it’s definitely not a rabbit hole developers go down happily.
- kps 3y ago
  > Terminal multiplexers like tmux need to create a pseudo-TTYs otherwise they couldn’t multiplex your terminal. [Edit: I misread the parent comment as saying multiplexers needed to do terminal emulation. Dtach only multiplexes (using pesudo-TTYs, of course) but unlike screen or tmux does not do terminal emulation. Original comment follows:] That isn't strictly true; the one I use (dtach) is just a multiplexer.
  - hnlmorg 3y ago
    dtach creates pseudo-TTYs too: - https://github.com/crigler/dtach/blob/master/master.c#L53 https://github.com/crigler/dtach/blob/master/master.c#L53 - https://github.com/crigler/dtach/blob/master/master.c#L108 https://github.com/crigler/dtach/blob/master/master.c#L108 As an aside, that projects source is surprisingly easy to follow. I was able to grok it in just a couple of minutes. Kudos to the author.
    - kps 3y ago
      Yes, I misread your comment, sorry, having had my mind stuck on terminal emulation.
      - hnlmorg 3y ago
        A PTY is an intrinsic part of terminal emulation. I think what you’re alluding to is that tmux gobbles up escape sequences (which is configurable by the way) and tracks cell positions. But as I said in another comment, the value add of tmux is its window management capabilities. That’s precisely why people like myself use it. And you cannot have that without complexity. In some edge cases that complexity causes issues, but on the whole I’m more productive for it. Saying it’s stupid that tmux does that is like saying it’s stupid that motorbikes have gears and fast patrol engines when a bmx works just as well. Sure, to someone who’s never seen a bike before they both look similar. But don’t be fooled by the fact that they both have 2 wheels because they cater to very different needs, hence their very different engineering designs.

foresto 3y ago

> In practice several of those ANSI escape code behaviors are hard-coded The most common examples I see these days are for red/green/yellow status text in shell scripts. To anyone planning to do that, or even more complex stuff, please consider using tput instead. tput setaf 1; echo this is red tput setaf 2; echo this is green tput setaf 3; echo this is yellow tput setaf 8; echo this is dim tput bold; echo this is bold tput sgr0; echo this is normal blue=$(tput setaf 4); normal=$(tput sgr0); echo ${blue}Color${normal} text. (man terminfo for arcane details.) It's wise to make sure output is going to a terminal before using colors, because they don't play well with logs or text processing commands. You can check with: test -t 1 If you use colors, please consider making them optional, since they don't work well for everyone or in every desktop color scheme. An environment variable makes a sensible switch.

kps 3y ago
> The most common examples I see these days are for red/yellow/green status text in shell scripts. Please don't do that without first checking the background colour. Yellow in particular on a light background is unreadable. (On an xterm-compatible terminal you can get the current background colour with OSC 11 ; ? BEL. Some popular terminals lie about being xterm-compatible, though.)
LinuxBender 3y ago
Make sure one has ncurses installed before switching to tput. Alpine Linux does not install this by default for what it's worth.
- foresto 3y ago
  # No tput, no cry: red=$(tput setaf 1 2>/dev/null || true) echo ${red}text
  - LinuxBender 3y ago
    You tryin' to get Bob Marley stuck in my head? Because it worked.
- 3y ago

kps 3y ago

Several popular terminals default to calling themselves xterm (e.g. `TERM=xterm-256color`), but are not xterm-compatible, and ignore bug reports (*cough* GNOME *cough*).

audidude 3y ago
Interesting. I don't work on VTE but just landed dozens of patches with no issue at all from pleasant maintainers.