4 ms·

Terminal control could (and should) have been out of band, e.g. via ioctls on the fds of the tty between shell and subprocess, instead of in-band via "special c

Terminal control could (and should) have been out of band, e.g. via ioctls on the fds of the tty between shell and subprocess, instead of in-band via "special characters", which usually (always?) hint at bad design (compare e.g. with SQL injection). As an example on how to do it right, consider how commands like "git diff" determine whether to paginate/colorize their output: they call the isatty(fd) stdlib function on stdout, which uses ioctl TIOCGETA to get terminal properties. Iff that fails, there's no TTY on the other end of the fd and "git diff" just writes the raw output. If only there even was a clean, consistently implemented, well-documented protocol... but there is not - few tools properly escape their output, and you can't get much right without a terminfo database, even with you can't get consistent results, and the protocol is a mess.

jstimpfle 10y ago
OOB is almost always a terrible idea. Urgent data like interrupts, as used in TCP (basically Unix signals over network), might be an exception. IOCTL's are a pain. Those who don't understand Unix are doomed to reinvent it, poorly. You should try to implement it the way you think it should be. The problem you will notice: "content" and "presentation" can't be easily separated because it's important when changes in presentation take effect. Also one man's presentation is the next man's content. They are just not independent streams, period. It's actually great that you can capture streams including control characters and replay them later. It works pretty well, especially these days where there aren't many competing instruction sets anymore. Even in HTML you need to tag elements with class names inline. (And the HTML approach could never work for general purpose terminals -- as opposed to crippled, specialized ones, like Mathematica / Jupyter notebooks). -- > If only there even was a clean, consistently implemented, well-documented protocol... but there is not - few tools properly escape their output, and you can't get much right without a terminfo database, even with you can't get consistent results, and the protocol is a mess. You can go GUI. But many of the programs you want to use aren't available for GUI, guess why... isatty() is a hack, but it's convenient and has very low false positives. If you filter output into another program before sending it to the terminal, the hack falls flat. You then need to provide a means for the user to be explicit, like `git diff --color=always`.
- Ao7bei3s 10y ago
  > Those who don't understand Unix are doomed to reinvent it, poorly. Let's avoid hollow arguments to authority that preclude any interesting criticism or discussion. Unix has its fair share of flaws. > it's important when changes in presentation take effect. [...] They are just not independent streams, period. Not independent, but separate. Your argument would also apply to stdout and stderr - these are separate even though not independent. ioctls could very well apply to a specific position in the data stream instead of being immediate. And if you don't like ioctls, there could well be a third output fd instead. In that case, the synchronization problem could be solved with a "barrier" feature. Can we at least agree that the actual terminal protocol situation as it is now quite frankly sucks? The poor documentation and completely inconsistent implementations have caused me quite some stress.
  - jstimpfle 10y ago
    > Let's avoid hollow arguments to authority that preclude any interesting criticism or discussion. It's not an argument. It's just a category for the rest of the comment. > Not independent, but separate. Your argument would also apply to stdout and stderr - these are separate even though not independent. No - technically they are completely independent. There is no ordering or interleaving or association defined. Actually stdin/out/err are nothing but conventions introduced by the shell. > ioctls could very well apply to a specific position in the data stream instead of being immediate. The general idea of a stream does not include positions - they don't have a clearly defined beginning or end. Streams are not like files on a filesystem, which do indeed have a size and can be indexed. Stream positions are another specialist concept that eases development for some apps or environments, but gets in the way of development of general purpose environments. > Can we at least agree that the actual terminal protocol situation as it is now quite frankly sucks? The poor documentation and completely inconsistent implementations have caused me quite some stress. Frankly there are not many problems with Unix terminals. They are quite simple. As Unix in general they just won't keep you from shooting yourself in the foot. Job control is hairy, though, but mostly because it's an ill-defined problem. In particular there was never a good agreed on mechanism for starting processes in independent environments. Systemd might have solved that to some degree (not a systemd fan). The situation with incompatible instruction sets (escase sequences) has also dramatically become better, partly because there are libraries like ncurses, partly because pretty much everything is VT100 compatible these days, partly because we have GUIs for complex graphical specialist applications. If you have a concrete problem - get in touch with me and I am happy to help.
kps 10y ago
> out of band, e.g. via ioctls You seem to be assuming terminals designed to support Unix, when in fact it's the other way around. None of this stuff is Unix-specific; it's just ISO 6429 / ISO 2022 / ASCII (or ECMA-48, ECMA-35, ECMA-6 if you want free copies). Out of band? Do you suddenly double everyone's wiring costs? That won't fly. How about multiplexing control and data streams onto the same channel? Yeah, that's exactly what ISO 6429 / ISO 2022 / ASCII does. If you want to express terminal control using function calls rather than inserting raw multiplexed control sequences into text — Yeah, that's a good idea… which Unix has had since 1978.
- jstimpfle 10y ago
  I'm curious, what exactly do you mean (1978)?
  - kps 10y ago
    Termlib (tgoto(), tputs(), tgetent()) is that old.

jstimpfle 10y ago

OOB is almost always a terrible idea. Urgent data like interrupts, as used in TCP (basically Unix signals over network), might be an exception. IOCTL's are a pain. Those who don't understand Unix are doomed to reinvent it, poorly. You should try to implement it the way you think it should be. The problem you will notice: "content" and "presentation" can't be easily separated because it's important when changes in presentation take effect. Also one man's presentation is the next man's content. They are just not independent streams, period. It's actually great that you can capture streams including control characters and replay them later. It works pretty well, especially these days where there aren't many competing instruction sets anymore. Even in HTML you need to tag elements with class names inline. (And the HTML approach could never work for general purpose terminals -- as opposed to crippled, specialized ones, like Mathematica / Jupyter notebooks). -- > If only there even was a clean, consistently implemented, well-documented protocol... but there is not - few tools properly escape their output, and you can't get much right without a terminfo database, even with you can't get consistent results, and the protocol is a mess. You can go GUI. But many of the programs you want to use aren't available for GUI, guess why... isatty() is a hack, but it's convenient and has very low false positives. If you filter output into another program before sending it to the terminal, the hack falls flat. You then need to provide a means for the user to be explicit, like `git diff --color=always`.

Ao7bei3s 10y ago
> Those who don't understand Unix are doomed to reinvent it, poorly. Let's avoid hollow arguments to authority that preclude any interesting criticism or discussion. Unix has its fair share of flaws. > it's important when changes in presentation take effect. [...] They are just not independent streams, period. Not independent, but separate. Your argument would also apply to stdout and stderr - these are separate even though not independent. ioctls could very well apply to a specific position in the data stream instead of being immediate. And if you don't like ioctls, there could well be a third output fd instead. In that case, the synchronization problem could be solved with a "barrier" feature. Can we at least agree that the actual terminal protocol situation as it is now quite frankly sucks? The poor documentation and completely inconsistent implementations have caused me quite some stress.
- jstimpfle 10y ago
  > Let's avoid hollow arguments to authority that preclude any interesting criticism or discussion. It's not an argument. It's just a category for the rest of the comment. > Not independent, but separate. Your argument would also apply to stdout and stderr - these are separate even though not independent. No - technically they are completely independent. There is no ordering or interleaving or association defined. Actually stdin/out/err are nothing but conventions introduced by the shell. > ioctls could very well apply to a specific position in the data stream instead of being immediate. The general idea of a stream does not include positions - they don't have a clearly defined beginning or end. Streams are not like files on a filesystem, which do indeed have a size and can be indexed. Stream positions are another specialist concept that eases development for some apps or environments, but gets in the way of development of general purpose environments. > Can we at least agree that the actual terminal protocol situation as it is now quite frankly sucks? The poor documentation and completely inconsistent implementations have caused me quite some stress. Frankly there are not many problems with Unix terminals. They are quite simple. As Unix in general they just won't keep you from shooting yourself in the foot. Job control is hairy, though, but mostly because it's an ill-defined problem. In particular there was never a good agreed on mechanism for starting processes in independent environments. Systemd might have solved that to some degree (not a systemd fan). The situation with incompatible instruction sets (escase sequences) has also dramatically become better, partly because there are libraries like ncurses, partly because pretty much everything is VT100 compatible these days, partly because we have GUIs for complex graphical specialist applications. If you have a concrete problem - get in touch with me and I am happy to help.

kps 10y ago

> out of band, e.g. via ioctls You seem to be assuming terminals designed to support Unix, when in fact it's the other way around. None of this stuff is Unix-specific; it's just ISO 6429 / ISO 2022 / ASCII (or ECMA-48, ECMA-35, ECMA-6 if you want free copies). Out of band? Do you suddenly double everyone's wiring costs? That won't fly. How about multiplexing control and data streams onto the same channel? Yeah, that's exactly what ISO 6429 / ISO 2022 / ASCII does. If you want to express terminal control using function calls rather than inserting raw multiplexed control sequences into text — Yeah, that's a good idea… which Unix has had since 1978.

jstimpfle 10y ago
I'm curious, what exactly do you mean (1978)?
- kps 10y ago
  Termlib (tgoto(), tputs(), tgetent()) is that old.