3 ms·

One thing that the author doesn't seem to know about is /dev/tty. In the article the escape codes are just sent to stdout. Though an awful lot of applications (

by _rmcx 4y ago

One thing that the author doesn't seem to know about is /dev/tty. In the article the escape codes are just sent to stdout. Though an awful lot of applications (including the greatest ones) do this, IMHO this is wrong. The terminal control codes are used to control a terminal, and they are often not meant to be part of the output stream, for example when the output is piped or redirected to a text file. When what you intend to do is to make your output fancy only when the output is a terminal, surely you should just send everything to stdout and use isatty to decide whether to also send those terminal control codes. But if what you want to build is a whole TUI like vim or the author's example app, you should send all the control to /dev/tty. This way if needed you can extend your app to be able to be a useful part of a pipe, as bytes sent to /dev/tty will not be redirected but will always be handled by the terminal. To prove that this can be useful, fzf, a fuzzy matcher, uses a TUI to let users input the pattern and pick among the matches, and prints the result to stdout. Sadly, however, it uses stderr to control the terminal instead of /dev/tty; this makes its ability to print error messages somewhat limited and its behavior unexpected when stderr is redirected. Also imagine that you can use vim in a pipe, instead of sed or awk, to see the effect of your edits live. Also, try `vi > /dev/null`. I'd say the behavior is a bug. IIRC ncurses makes use of /dev/tty and by default makes the app made with it redirectable, and this is a reason we should use it in the 21st century, among others. What's sad to me is that so far all the Rust terminal libs I've seen ignore /dev/tty, so it's impossible to use them to build something that both have a good TUI and can be used in pipes.

knorker 4y ago
So what happens when you do "./foo | less"? You want "foo" to start injecting formatting to the terminal, while "less" is too?
rcfox 4y ago
This is my first time hearing of this functionality. How does it work with buffered stdout? For example, if you wanted to colour a single word?
- _rmcx 4y ago
  > This is my first time hearing of this functionality. I was also surprised when I discovered this. IMO this should be used everywhere but I've rarely seen it. > How does it work with buffered stdout? For example, if you wanted to colour a single word? This is not affected by buffering. What matters is what the output device/file/stream gets eventually. How coloring a single word works depends on how you use it. You can do write_and_flush_to_dev_tty(BEGIN_RED_TEXT); write_and_flush_to_dev_tty("word"); write_and_flush_to_dev_tty(END_RED_TEXT); The write_and_flush_... pseudo functions are written in this way just to make it clear that I'm describing the behavior when the output gets the bytes immediately. I don't mean that you should use /dev/tty like this. Redirection won't have any effect on these lines of code. You always get a red "word" on the terminal. The redirection target doesn't get anything. write_and_flush_to_dev_tty(BEGIN_RED_TEXT); write_and_flush_to_stdout("word"); write_and_flush_to_dev_tty(END_RED_TEXT); When directly used on a terminal, you get a red "word" on your terminal, but when you do redirection, the target only gets plaintext "word". write_and_flush_to_dev_tty(BEGIN_RED_TEXT); write_and_flush_to_stdout("word"); write_and_flush_to_stdout(END_RED_TEXT); When redirected, your terminal stays red after the red "word" is printed, and your redirection target gets an extra escape code. When stdout is a tty, /dev/tty is the same as stdout, so a write that goes to one goes to the other. In this case, even if you don't flush after write immediately it's likely not a problem. Still it's something that the developer should pay attention to. ------------- Edit: Please ignore all the pseudocode examples. They don't convey what I wanted to express. Just use /dev/tty in the same way as how you would use stdin/stdout/stderr that is connected to a terminal. Only difference is that it is a rw device that can be used for input and output at the same time.
  - hoseja 4y ago
    So, the answer to "How does it work with buffered stdout?" is, "It doesn't, you have to keep flushing."
  - geocar 4y ago
    > write_and_flush_to_dev_tty(BEGIN_RED_TEXT),write_and_flush_to_stdout("word"),write_and_flush_to_dev_tty(END_RED_TEXT); This isn't guaranteed to be serialized: Someone trying to log the output of your application might try: | tee /dev/tty | logger ... or they might be running under kubernetes/docker (which does much the same thing). I suggest the following: 1. If fd 0 and fd 1 are both ttys (isatty), and they point to the same tty (ttyname) then use it a tty (like your first example) 2. If fd 1 is not a tty, and fd 0 is a tty, write plaintext to fd 1. The user wants to filter output. 3. If neither fd 0 or 1 is a tty, write twice: do interactive stuff and send your vt-sequences with the text embedded to /dev/tty and a plaintext copy to fd 1. Now the user doesn't need the extra tee, and we no longer need to worry about synchronisation. Users don't typically (meaningfully) grep interactive applications, so I think it's a good compromise for interactive servers and tuis. But if you're not reading input or doing absolute cursor movement, and you just want some spicy log lines, I don't think you should be doing any of this fiddling: Just write sequences to stdout if it's a tty (or if the user specifically tells you to). I also recommend checking for $NO_COLOR in the environment and honouring the users wishes here: Some users are colourblind so this represents a real accessibility issue for them. One of the advantages of ncurses/terminfo is you get some accessibility features you might not have known you needed.
donatj 4y ago
A very nervous very young me gave a bad talk on terminals at a conference about 10 years ago. Reviews of my talk were pretty rough, but a number of them mentioned learning that they should have been sending their control codes to standard error, so it wasn't a total waste...
sph 4y ago
I loathe to imagine how those modern TUI libraries that have been popping up recently [1] that emulate Elm and React rendering in the terminal deal when isatty == false. We're basically recreating Flash for the terminal, where we got a fancy UI but we lose the original text functionality, i.e. output is non pipable anymore. Not ideal when working on UNIX systems. 1: https://news.ycombinator.com/item?id=31328205 https://news.ycombinator.com/item?id=31328205
jart 4y ago
Have you considered that many of us want ansi codes in the pipeline? For example, all the log viewers I use understand things like color codes. If I don't want ANSI codes in the output, then I can just pipe it through sed 's/\x1b\[[;[:digit:]]*m//g' which is easy. However if a program tries to be "smart" like you're describing w.r.t. hiding ANSI codes, then I have to go to all this trouble wrapping it inside another program which is a fake pseudo-terminal, which captures and extracts the real output.
- _rmcx 4y ago
  > Have you considered that many of us want ansi codes in the pipeline? Yes I have. Using /dev/tty doesn't stop an application's author from adding a --color that lets their program send color codes to stdout. But if the author doesn't use /dev/tty either stdout or stderr of their application can't be redirected.
  - jart 4y ago
    Yes but no one agrees on what that should be. For example, I need to flip through a 623 page manual to discover -fdiagnostic-color=always is the magic incantation for gcc. I have to repeat that for every app I use. Some do it using environment variables, except no one agrees on names there either, so I have to bloat my environment and it slows down process creation. Whereas with my sed solution, anyone who knows regex can could write it in a few minutes, and it only has to be figured out one time. Furthermore, there will eventually come a day when the tools we pipe stuff into all get good enough to gracefully consume ANSI codes, and there's no longer a need for sed. But we can't evolve in that direction if we use the solution you're proposing, which binds us to a colorless past. Erring on the side of having more information is always a good thing. The burden of the flag should be on disabling, not enabling, and there shouldn't even need to be a flag since there's sed.
    - Normal_gaussian 4y ago
      > man gcc Search color, its the first result as an abbreviated reference and the second as a full explanation. Its also the first result on google/ddg/etc. You have a good argument, but claiming you have to "flip through a 623 page manual" and "with my sed solution, anyone who knows regex could write it in a few minutes" detracts from your point.
trasz 4y ago
If you want to build a TUI, you absolutely shouldn’t try to mess with devices, nor should you assume your terminal is /dev/tty. You should be using isatty(3) libc function.
deleted 4y ago
[deleted]

by _rmcx 4y ago

knorker 4y ago

So what happens when you do "./foo | less"? You want "foo" to start injecting formatting to the terminal, while "less" is too?

rcfox 4y ago

This is my first time hearing of this functionality. How does it work with buffered stdout? For example, if you wanted to colour a single word?

_rmcx 4y ago
> This is my first time hearing of this functionality. I was also surprised when I discovered this. IMO this should be used everywhere but I've rarely seen it. > How does it work with buffered stdout? For example, if you wanted to colour a single word? This is not affected by buffering. What matters is what the output device/file/stream gets eventually. How coloring a single word works depends on how you use it. You can do write_and_flush_to_dev_tty(BEGIN_RED_TEXT); write_and_flush_to_dev_tty("word"); write_and_flush_to_dev_tty(END_RED_TEXT); The write_and_flush_... pseudo functions are written in this way just to make it clear that I'm describing the behavior when the output gets the bytes immediately. I don't mean that you should use /dev/tty like this. Redirection won't have any effect on these lines of code. You always get a red "word" on the terminal. The redirection target doesn't get anything. write_and_flush_to_dev_tty(BEGIN_RED_TEXT); write_and_flush_to_stdout("word"); write_and_flush_to_dev_tty(END_RED_TEXT); When directly used on a terminal, you get a red "word" on your terminal, but when you do redirection, the target only gets plaintext "word". write_and_flush_to_dev_tty(BEGIN_RED_TEXT); write_and_flush_to_stdout("word"); write_and_flush_to_stdout(END_RED_TEXT); When redirected, your terminal stays red after the red "word" is printed, and your redirection target gets an extra escape code. When stdout is a tty, /dev/tty is the same as stdout, so a write that goes to one goes to the other. In this case, even if you don't flush after write immediately it's likely not a problem. Still it's something that the developer should pay attention to. ------------- Edit: Please ignore all the pseudocode examples. They don't convey what I wanted to express. Just use /dev/tty in the same way as how you would use stdin/stdout/stderr that is connected to a terminal. Only difference is that it is a rw device that can be used for input and output at the same time.
- hoseja 4y ago
  So, the answer to "How does it work with buffered stdout?" is, "It doesn't, you have to keep flushing."
- geocar 4y ago
  > write_and_flush_to_dev_tty(BEGIN_RED_TEXT),write_and_flush_to_stdout("word"),write_and_flush_to_dev_tty(END_RED_TEXT); This isn't guaranteed to be serialized: Someone trying to log the output of your application might try: | tee /dev/tty | logger ... or they might be running under kubernetes/docker (which does much the same thing). I suggest the following: 1. If fd 0 and fd 1 are both ttys (isatty), and they point to the same tty (ttyname) then use it a tty (like your first example) 2. If fd 1 is not a tty, and fd 0 is a tty, write plaintext to fd 1. The user wants to filter output. 3. If neither fd 0 or 1 is a tty, write twice: do interactive stuff and send your vt-sequences with the text embedded to /dev/tty and a plaintext copy to fd 1. Now the user doesn't need the extra tee, and we no longer need to worry about synchronisation. Users don't typically (meaningfully) grep interactive applications, so I think it's a good compromise for interactive servers and tuis. But if you're not reading input or doing absolute cursor movement, and you just want some spicy log lines, I don't think you should be doing any of this fiddling: Just write sequences to stdout if it's a tty (or if the user specifically tells you to). I also recommend checking for $NO_COLOR in the environment and honouring the users wishes here: Some users are colourblind so this represents a real accessibility issue for them. One of the advantages of ncurses/terminfo is you get some accessibility features you might not have known you needed.

donatj 4y ago

A very nervous very young me gave a bad talk on terminals at a conference about 10 years ago. Reviews of my talk were pretty rough, but a number of them mentioned learning that they should have been sending their control codes to standard error, so it wasn't a total waste...

sph 4y ago

I loathe to imagine how those modern TUI libraries that have been popping up recently [1] that emulate Elm and React rendering in the terminal deal when isatty == false. We're basically recreating Flash for the terminal, where we got a fancy UI but we lose the original text functionality, i.e. output is non pipable anymore. Not ideal when working on UNIX systems. 1: https://news.ycombinator.com/item?id=31328205 https://news.ycombinator.com/item?id=31328205

jart 4y ago

Have you considered that many of us want ansi codes in the pipeline? For example, all the log viewers I use understand things like color codes. If I don't want ANSI codes in the output, then I can just pipe it through sed 's/\x1b\[[;[:digit:]]*m//g' which is easy. However if a program tries to be "smart" like you're describing w.r.t. hiding ANSI codes, then I have to go to all this trouble wrapping it inside another program which is a fake pseudo-terminal, which captures and extracts the real output.

_rmcx 4y ago
> Have you considered that many of us want ansi codes in the pipeline? Yes I have. Using /dev/tty doesn't stop an application's author from adding a --color that lets their program send color codes to stdout. But if the author doesn't use /dev/tty either stdout or stderr of their application can't be redirected.
- jart 4y ago
  Yes but no one agrees on what that should be. For example, I need to flip through a 623 page manual to discover -fdiagnostic-color=always is the magic incantation for gcc. I have to repeat that for every app I use. Some do it using environment variables, except no one agrees on names there either, so I have to bloat my environment and it slows down process creation. Whereas with my sed solution, anyone who knows regex can could write it in a few minutes, and it only has to be figured out one time. Furthermore, there will eventually come a day when the tools we pipe stuff into all get good enough to gracefully consume ANSI codes, and there's no longer a need for sed. But we can't evolve in that direction if we use the solution you're proposing, which binds us to a colorless past. Erring on the side of having more information is always a good thing. The burden of the flag should be on disabling, not enabling, and there shouldn't even need to be a flag since there's sed.
  - Normal_gaussian 4y ago
    > man gcc Search color, its the first result as an abbreviated reference and the second as a full explanation. Its also the first result on google/ddg/etc. You have a good argument, but claiming you have to "flip through a 623 page manual" and "with my sed solution, anyone who knows regex could write it in a few minutes" detracts from your point.

trasz 4y ago

If you want to build a TUI, you absolutely shouldn’t try to mess with devices, nor should you assume your terminal is /dev/tty. You should be using isatty(3) libc function.

deleted 4y ago

[deleted]