8 ms·

Raw keyboard handling in Unix terminals

115 points by drran 5y ago

voldacar 5y ago
This feels like the computing equivalent of epicycles in medieval physics. Will we ever just admit that the vt100 tty is just no longer suitable as a basic OS abstraction, and needs to be torn out and replaced? Or in 50 years will simple interactive programs still not be able to detect keyup without importing a bunch of bloat from x11/wayland?
- tyingq 5y ago
  I suspect in 50 years that we will have lost control of the local compute end of things. Our new "dumb client" will be some future iteration of a web browser. And you won't be allowed to run anything outside of it. You'll be able to see keyup, but most of everything else will be on somebody else's server.
  - jrumbut 5y ago
    Over the last 50 years there have been a few cycles between thin and thick clients (in different contexts) so hopefully it's not all one direction from here.
    - tyingq 5y ago
      I agree, though this time there's big money behind taking local control away.
      - generalizations 5y ago
        Which is why, as long as there's hackers, there'll be another cycle in the other direction.
        tyingq 5y ago
        They aren't having much luck with the iPhone...
        w0de0 5y ago
        The are! It's just not an everyday occurrence - and so is worth much more - so most zero days end up in nation state hands.
        kingofclams 5y ago
        Take what i say with a grain of salt, (I’m not an iPhone hacker and I follow their work semi-loosely but from my understanding there are a number of available and documented tethered exploits. I wouldn’t doubt theres a number of untethered exploits that are waiting in Apple’s bug tracker system, or are simply too abusable for the general public to know about.
    - agumonkey 5y ago
      I wonder if these cycles have been studied.
    - saurik 5y ago
      The thick clients, though, seem heavily tied to mechanisms that still prevent local control.
- sprash 5y ago
  If anything the vt100 ttys are just as suitable as ever. Command line interfaces are superior even today because the step from interaction to automation is as minimal as possible. I predict that the futuristc VR GUI of tomorrow will still be just a bunch of vt100 terminals floating in space.
  - voldacar 5y ago
    Can't we dream bigger though? For instance, I feel like templeOS's command line is unironically conceptually superior than the linux commandline or any other unix. I should be able to mix text with 3d models or images or sound, pipe them between programs, etc. I should be able to pause a running program, save it to disk, and resume it later. There are all sorts of things that I should be able to do but can't, because of the deliberate unix fetishization of 1970s era computing. >Command line interfaces are superior even today because the step from interaction to automation is as minimal as possible. Agreed completely. In fact, the OS should force GUI programs to all speak a coherent language so that you can automate them and pipe data between them. And if the machine is going to serve the user, the user should be in control of the GUI, they should be able to move buttons and UI elements around into the configuration they prefer. This would only be possible if GUI programs had minimal control over actual UI layout, which would make webdevs angry. But given the state of web UI, I think that's an acceptable sacrifice. Much of the above was possible in lisp machines. But sadly I don't think it would be possible to create such an OS today (unless the "OS" is running in a linux VM and using linux for the actual hardware interface) due to the sheer diversity of hardware. And the closed source nvidia driver would ensure that you never got cool graphics.
    - sprash 5y ago
      In my opinion the mistake was to move away from the terminal in the first place. Many gui commands like "draw button in bottom right corner" could be done via escape codes and creating a gui program would be as simple as writing a bunch of commands to stdout. As far as I know the earliest prototypes of X worked just like that.
      - 5y ago
- otabdeveloper4 5y ago
  > no longer suitable No way, it's way more popular and useful than ever. We just need to ditch vt100 and settle on some sort of xterm variation as the new standard.
  - blueflow 5y ago
    This won't have much effect, since xterm is an DEC VT emulator.
    - nitrogen 5y ago
      A superset though. It's perfectly fine to support ANSI and VT-100 if we can add new escape codes to switch to a standardized raw mode, just like has been done for 8-bit and 24-bit color, and for mouse support.
- rbanffy 5y ago
  If I read this correctly, this proposal aims to solve just that kind of stuff while remaining compatible with all the legacy software in use today.
Conlectus 5y ago
Excellent to see, the ESC ambiguity problem is one that has especially bugged me. I hope there's a plan to standardize this with the relevant organizations! I worry that's the only way to see widespread adoption.
- nine_k 5y ago
  Yes, I can imagine a mostly-backwards-compatible, but much improved, standard; let's call it VT-2021.
  - drran 5y ago
    If you are native English speaker, then start with kind of "caniuse" site, but for terminals and VT, please. It will help a lot if we will be just aware of available features in existing terminals and their compatibility with standards.
    - nine_k 5y ago
      Not a web site, but terninfo and termcap databases, normally present on a Unix system, have this information. I would hazard to say that some basic subset of VT100 commands is "universally" supported, even on Windows if one uses Windows Terminal. Various obscure high-end features from 1980s, like programmable font bitmaps, or sixel support, much less so. Also, modern terminals support Unicode, including emoji (as double-width symbols), which was unimaginable in 1980s or 1990s.
      - DHowett 5y ago
        > if one uses Windows Terminal. This is also true of the Windows Console as of Windows 10. Any VT100 (also VT220, VT4xx, VT5xx, xterm ...) support we add to Terminal eventually flows out to the console host integrated into Windows. :)
      - drran 5y ago
        How to display terminfo in a consumable way? What about non-standard features, like drawing, sixels, sound, mouse, touches, clipboard, fonts, true color, transparent background, color emoji, double size text, etc.?
gonzus 5y ago
This is awesome. For a long while I have wanted to write character-based applications with greater control of the keyboard; think "a terminal tetris that can recognize key combinations such as Alt-I, Ctrl-J, Alt-Ctrl-PgUp, Meta-Ctrl-F14, etc." This should work OK whether running locally on the console, on a (serial) terminal, on a graphical (X Windows / Wayland) terminal, on a remote (SSH) terminal, etc. I wish something like this was also created for the mouse, so that you could use your mouse everywhere (including the console without any graphical environment running), remotely over ssh, etc. I see no reason why a similar protocol could not transmit mouse events over ssh and allow you to interact, using your mouse, with a remote character-based application. I would love for such proposals to gain traction and be accepted everywhere.
- sprash 5y ago
  There are already several standardized mouse protocols available some even with pixel precision. The most complete implementation of those protocols has xterm. They can be tested with a program called "vttest".
- smhenderson 5y ago
  Have you heard of gpm[1] - general purpose mouse? Slackware offers to install and enable it by default, even these days. It's a terminal based mouse server. ncurses supports it so it's theoretically possible to do all the same things in a TUI that you would in a GUI with a mouse. [1] https://en.wikipedia.org/wiki/GPM_(software) https://en.wikipedia.org/wiki/GPM_(software)
  - drran 5y ago
    GPM is for console, not for terminal. I used it a lot in virtual console, before switching to X11 based terminal emulators.
    - deleted 5y ago
      [deleted]
softblush 5y ago
The usual name clash rant. Why kitty? KiTTY is a fork of PuTTY and has been around since at least ~2007 or so
- jrhawley 5y ago
  This is written by the KiTTY developer. My guess is that when you open up a kitty session, you're not trying to open a subsequent one inside. So running `kitty` inside a session will give you a preview of the new protocol being proposed here
  - nitrogen 5y ago
    Are you certain? This looks like it's written by the author of the newer kitty terminal emulator, not the older KiTTY ssh/telnet client derived from PuTTY.
    - jrhawley 5y ago
      You know what, you're right. It's the terminal emulator, not the putty fork. I guess my mistake proves softblush's point
mongol 5y ago
"Emit the escape sequence CSI < u at application exit or just before leaving alternate screen mode to restore the previously used keyboard mode." What happens if the application crashes without emitting this? The user is dropped in to the shell and the terminal and shell might misbehave together?
- amadeusz 5y ago
  I guess same thing that happens to any application that sets weird modes. You should use 'reset' command which just emits escape code to reset terminal and terminal emulator hopefully sets everything back correctly.
  - tedunangst 5y ago
    If the application switched to key press mode, you're not going to be able to type reset.
    - amadeusz 5y ago
      Well, from what I've read all alphanumeric can be typed just fine, they go as is to terminal, problem is with special keys that some of them emit escape sequences that confuse shell or combinations of those. But not to trust what I just read I've installed kitty and tested it and you can type in the "reset" just fine. I hit a bit of a problem with "Entering" it, as apparently if you have numlock enabled (which I do on login) pressing "Enter" sends escape code: "^[[13;129u", but if you disable numlock it is typed in as per documentation: "0x0d - for the Enter key", which in any ascii table is Carriage Return. And yes, issuing reset restored normal terminal operation.
      - tedunangst 5y ago
        If you enable all key events there's going to be a "key release" event between the r and e and all subsequent letters of reset.
        5y ago
amadeusz 5y ago
The thing I would like to see is this making it into terminfo entries, meaning that it is possible to detect that terminal supports this behaviour, otherwise while great it is terminal emulator specific.
amelius 5y ago
This should be an RFC.
- rbanffy 5y ago
  How hard would it be to turn this into one?
  - drran 5y ago
    There is already a de facto standard for extended codes: Xterm's modifyOtherKeys. Applications: https://vimhelp.org/map.txt.html#modifyOtherKeys https://vimhelp.org/map.txt.html#modifyOtherKeys https://github.com/emacs-mirror/emacs/blob/d68f2b8681f8eeb6bbf1b4476a88f00b2962179e/lisp/term/xterm.el#L35 https://github.com/emacs-mirror/emacs/blob/d68f2b8681f8eeb6b... https://github.com/akinomyoga/ble.sh https://github.com/akinomyoga/ble.sh https://github.com/mawww/kakoune/commit/47c0d2038807757207cde3b3e00fbc1ca9448178 https://github.com/mawww/kakoune/commit/47c0d2038807757207cd... https://github.com/tmux/tmux/wiki/Modifier-Keys#extended-keys https://github.com/tmux/tmux/wiki/Modifier-Keys#extended-key... https://github.com/magiblot/tvision/commit/a50569f549465c9eda245eeb714e1e805fa2cf88 https://github.com/magiblot/tvision/commit/a50569f549465c9ed... Terminals: xterm (obviously) https://github.com/gnachman/iTerm2/commit/ac1aa575011028ed86851c15b177b0e9c78082e0 https://github.com/gnachman/iTerm2/commit/ac1aa575011028ed86... https://github.com/r-c-f/foot https://github.com/r-c-f/foot https://github.com/kovidgoyal/kitty/commit/d360d077d1f7723b1023b77969c38436353b9519 https://github.com/kovidgoyal/kitty/commit/d360d077d1f7723b1... https://github.com/tomszilagyi/zutty https://github.com/tomszilagyi/zutty https://github.com/mintty/mintty/blob/master/wiki/CtrlSeqs.md https://github.com/mintty/mintty/blob/master/wiki/CtrlSeqs.m...
Sniffnoy 5y ago
Question: How does this differ from Paul Evans's solution in libtickit, as discussed here[0]? (Yeah, yeah, I can read the whole thing and see for myself, but I'm hoping someone has a quick answer.) [0] http://www.leonerd.org.uk/hacks/fixterms/ http://www.leonerd.org.uk/hacks/fixterms/
- aumerle 5y ago
  The first couple of parapgraphs tell you that it is based on that and list all the issues with fixterms that it fixes.
  - Sniffnoy 5y ago
    I see, thanks! I missed that.

115 points by drran 5y ago

This feels like the computing equivalent of epicycles in medieval physics. Will we ever just admit that the vt100 tty is just no longer suitable as a basic OS abstraction, and needs to be torn out and replaced? Or in 50 years will simple interactive programs still not be able to detect keyup without importing a bunch of bloat from x11/wayland?

tyingq 5y ago
I suspect in 50 years that we will have lost control of the local compute end of things. Our new "dumb client" will be some future iteration of a web browser. And you won't be allowed to run anything outside of it. You'll be able to see keyup, but most of everything else will be on somebody else's server.
- jrumbut 5y ago
  Over the last 50 years there have been a few cycles between thin and thick clients (in different contexts) so hopefully it's not all one direction from here.
  - tyingq 5y ago
    I agree, though this time there's big money behind taking local control away.
    - generalizations 5y ago
      Which is why, as long as there's hackers, there'll be another cycle in the other direction.
      - tyingq 5y ago
        They aren't having much luck with the iPhone...
        w0de0 5y ago
        The are! It's just not an everyday occurrence - and so is worth much more - so most zero days end up in nation state hands.
        kingofclams 5y ago
        Take what i say with a grain of salt, (I’m not an iPhone hacker and I follow their work semi-loosely but from my understanding there are a number of available and documented tethered exploits. I wouldn’t doubt theres a number of untethered exploits that are waiting in Apple’s bug tracker system, or are simply too abusable for the general public to know about.
  - agumonkey 5y ago
    I wonder if these cycles have been studied.
  - saurik 5y ago
    The thick clients, though, seem heavily tied to mechanisms that still prevent local control.
sprash 5y ago
If anything the vt100 ttys are just as suitable as ever. Command line interfaces are superior even today because the step from interaction to automation is as minimal as possible. I predict that the futuristc VR GUI of tomorrow will still be just a bunch of vt100 terminals floating in space.
- voldacar 5y ago
  Can't we dream bigger though? For instance, I feel like templeOS's command line is unironically conceptually superior than the linux commandline or any other unix. I should be able to mix text with 3d models or images or sound, pipe them between programs, etc. I should be able to pause a running program, save it to disk, and resume it later. There are all sorts of things that I should be able to do but can't, because of the deliberate unix fetishization of 1970s era computing. >Command line interfaces are superior even today because the step from interaction to automation is as minimal as possible. Agreed completely. In fact, the OS should force GUI programs to all speak a coherent language so that you can automate them and pipe data between them. And if the machine is going to serve the user, the user should be in control of the GUI, they should be able to move buttons and UI elements around into the configuration they prefer. This would only be possible if GUI programs had minimal control over actual UI layout, which would make webdevs angry. But given the state of web UI, I think that's an acceptable sacrifice. Much of the above was possible in lisp machines. But sadly I don't think it would be possible to create such an OS today (unless the "OS" is running in a linux VM and using linux for the actual hardware interface) due to the sheer diversity of hardware. And the closed source nvidia driver would ensure that you never got cool graphics.
  - sprash 5y ago
    In my opinion the mistake was to move away from the terminal in the first place. Many gui commands like "draw button in bottom right corner" could be done via escape codes and creating a gui program would be as simple as writing a bunch of commands to stdout. As far as I know the earliest prototypes of X worked just like that.
    - 5y ago
otabdeveloper4 5y ago
> no longer suitable No way, it's way more popular and useful than ever. We just need to ditch vt100 and settle on some sort of xterm variation as the new standard.
- blueflow 5y ago
  This won't have much effect, since xterm is an DEC VT emulator.
  - nitrogen 5y ago
    A superset though. It's perfectly fine to support ANSI and VT-100 if we can add new escape codes to switch to a standardized raw mode, just like has been done for 8-bit and 24-bit color, and for mouse support.
rbanffy 5y ago
If I read this correctly, this proposal aims to solve just that kind of stuff while remaining compatible with all the legacy software in use today.

Conlectus 5y ago

Excellent to see, the ESC ambiguity problem is one that has especially bugged me. I hope there's a plan to standardize this with the relevant organizations! I worry that's the only way to see widespread adoption.

nine_k 5y ago
Yes, I can imagine a mostly-backwards-compatible, but much improved, standard; let's call it VT-2021.
- drran 5y ago
  If you are native English speaker, then start with kind of "caniuse" site, but for terminals and VT, please. It will help a lot if we will be just aware of available features in existing terminals and their compatibility with standards.
  - nine_k 5y ago
    Not a web site, but terninfo and termcap databases, normally present on a Unix system, have this information. I would hazard to say that some basic subset of VT100 commands is "universally" supported, even on Windows if one uses Windows Terminal. Various obscure high-end features from 1980s, like programmable font bitmaps, or sixel support, much less so. Also, modern terminals support Unicode, including emoji (as double-width symbols), which was unimaginable in 1980s or 1990s.
    - DHowett 5y ago
      > if one uses Windows Terminal. This is also true of the Windows Console as of Windows 10. Any VT100 (also VT220, VT4xx, VT5xx, xterm ...) support we add to Terminal eventually flows out to the console host integrated into Windows. :)
    - drran 5y ago
      How to display terminfo in a consumable way? What about non-standard features, like drawing, sixels, sound, mouse, touches, clipboard, fonts, true color, transparent background, color emoji, double size text, etc.?

gonzus 5y ago

This is awesome. For a long while I have wanted to write character-based applications with greater control of the keyboard; think "a terminal tetris that can recognize key combinations such as Alt-I, Ctrl-J, Alt-Ctrl-PgUp, Meta-Ctrl-F14, etc." This should work OK whether running locally on the console, on a (serial) terminal, on a graphical (X Windows / Wayland) terminal, on a remote (SSH) terminal, etc. I wish something like this was also created for the mouse, so that you could use your mouse everywhere (including the console without any graphical environment running), remotely over ssh, etc. I see no reason why a similar protocol could not transmit mouse events over ssh and allow you to interact, using your mouse, with a remote character-based application. I would love for such proposals to gain traction and be accepted everywhere.

sprash 5y ago
There are already several standardized mouse protocols available some even with pixel precision. The most complete implementation of those protocols has xterm. They can be tested with a program called "vttest".
smhenderson 5y ago
Have you heard of gpm[1] - general purpose mouse? Slackware offers to install and enable it by default, even these days. It's a terminal based mouse server. ncurses supports it so it's theoretically possible to do all the same things in a TUI that you would in a GUI with a mouse. [1] https://en.wikipedia.org/wiki/GPM_(software) https://en.wikipedia.org/wiki/GPM_(software)
- drran 5y ago
  GPM is for console, not for terminal. I used it a lot in virtual console, before switching to X11 based terminal emulators.
  - deleted 5y ago
    [deleted]

softblush 5y ago

The usual name clash rant. Why kitty? KiTTY is a fork of PuTTY and has been around since at least ~2007 or so

jrhawley 5y ago
This is written by the KiTTY developer. My guess is that when you open up a kitty session, you're not trying to open a subsequent one inside. So running `kitty` inside a session will give you a preview of the new protocol being proposed here
- nitrogen 5y ago
  Are you certain? This looks like it's written by the author of the newer kitty terminal emulator, not the older KiTTY ssh/telnet client derived from PuTTY.
  - jrhawley 5y ago
    You know what, you're right. It's the terminal emulator, not the putty fork. I guess my mistake proves softblush's point

mongol 5y ago

"Emit the escape sequence CSI < u at application exit or just before leaving alternate screen mode to restore the previously used keyboard mode." What happens if the application crashes without emitting this? The user is dropped in to the shell and the terminal and shell might misbehave together?

amadeusz 5y ago
I guess same thing that happens to any application that sets weird modes. You should use 'reset' command which just emits escape code to reset terminal and terminal emulator hopefully sets everything back correctly.
- tedunangst 5y ago
  If the application switched to key press mode, you're not going to be able to type reset.
  - amadeusz 5y ago
    Well, from what I've read all alphanumeric can be typed just fine, they go as is to terminal, problem is with special keys that some of them emit escape sequences that confuse shell or combinations of those. But not to trust what I just read I've installed kitty and tested it and you can type in the "reset" just fine. I hit a bit of a problem with "Entering" it, as apparently if you have numlock enabled (which I do on login) pressing "Enter" sends escape code: "^[[13;129u", but if you disable numlock it is typed in as per documentation: "0x0d - for the Enter key", which in any ascii table is Carriage Return. And yes, issuing reset restored normal terminal operation.
    - tedunangst 5y ago
      If you enable all key events there's going to be a "key release" event between the r and e and all subsequent letters of reset.
      - 5y ago

amadeusz 5y ago

The thing I would like to see is this making it into terminfo entries, meaning that it is possible to detect that terminal supports this behaviour, otherwise while great it is terminal emulator specific.

amelius 5y ago

This should be an RFC.

rbanffy 5y ago
How hard would it be to turn this into one?
- drran 5y ago
  There is already a de facto standard for extended codes: Xterm's modifyOtherKeys. Applications: https://vimhelp.org/map.txt.html#modifyOtherKeys https://vimhelp.org/map.txt.html#modifyOtherKeys https://github.com/emacs-mirror/emacs/blob/d68f2b8681f8eeb6bbf1b4476a88f00b2962179e/lisp/term/xterm.el#L35 https://github.com/emacs-mirror/emacs/blob/d68f2b8681f8eeb6b... https://github.com/akinomyoga/ble.sh https://github.com/akinomyoga/ble.sh https://github.com/mawww/kakoune/commit/47c0d2038807757207cde3b3e00fbc1ca9448178 https://github.com/mawww/kakoune/commit/47c0d2038807757207cd... https://github.com/tmux/tmux/wiki/Modifier-Keys#extended-keys https://github.com/tmux/tmux/wiki/Modifier-Keys#extended-key... https://github.com/magiblot/tvision/commit/a50569f549465c9eda245eeb714e1e805fa2cf88 https://github.com/magiblot/tvision/commit/a50569f549465c9ed... Terminals: xterm (obviously) https://github.com/gnachman/iTerm2/commit/ac1aa575011028ed86851c15b177b0e9c78082e0 https://github.com/gnachman/iTerm2/commit/ac1aa575011028ed86... https://github.com/r-c-f/foot https://github.com/r-c-f/foot https://github.com/kovidgoyal/kitty/commit/d360d077d1f7723b1023b77969c38436353b9519 https://github.com/kovidgoyal/kitty/commit/d360d077d1f7723b1... https://github.com/tomszilagyi/zutty https://github.com/tomszilagyi/zutty https://github.com/mintty/mintty/blob/master/wiki/CtrlSeqs.md https://github.com/mintty/mintty/blob/master/wiki/CtrlSeqs.m...

Sniffnoy 5y ago

Question: How does this differ from Paul Evans's solution in libtickit, as discussed here[0]? (Yeah, yeah, I can read the whole thing and see for myself, but I'm hoping someone has a quick answer.) [0] http://www.leonerd.org.uk/hacks/fixterms/ http://www.leonerd.org.uk/hacks/fixterms/

aumerle 5y ago
The first couple of parapgraphs tell you that it is based on that and list all the issues with fixterms that it fixes.
- Sniffnoy 5y ago
  I see, thanks! I missed that.