10 ms·

What happens when you press a key in your terminal?

408 points by robenkleene 4y ago

ratsmack 4y ago
The picture of the VT100 brings back memories. I had one at home and would connect to our Vax 11/780 to program in FORTRAN using the EDT editor... using a 1200 baud modem. (about 1981 or so)
- bjelkeman-again 4y ago
  I wish I had kept my amber terminal. Great for working late at night, very focused. Having a modern display projecting amber colours just isn’t the same.
- lambic 4y ago
  I shared a flat in London with two other geeks in the 90s and we wired up vt100 terminals in each of our rooms so we could talk to each other without leaving our rooms. They didn't last long, they were quickly replaced by PCs on a lan so we could play Doom.
andreskytt 4y ago
And this, kids, is why it was possible to do fun things to computers of other people when their os did not tell the modem to stop accepting incoming bytes when the line was dropped and then picked up again. It just accepted bytes from wherever they might come…
oars 4y ago
So I didn't know that when I press backspace in the terminal, "x08" is being sent, not "^H". What is the purpose of caret notation then? Is it just for human readability? e.g. my terminal shows ^H sometimes when I press backspace
- yjftsjthsd-h 4y ago
  Pretty sure it's for the human, yes; how else would you represent "hey human, something just echoed the thing you get from ctrl-h"? (Keeping in mind that this almost certainly predates unicode and fancy fonts, and probably colors in the terminal. I'm open to the idea that we could do better today, but it's hard to overcome 50 years of tradition.)
- IncRnd 4y ago
  > So I didn't know that when I press backspace in the terminal, "x08" is being sent, not "^H". "0x08" is "^H". H is the eighth letter of the alphabet. There is no difference between those except for the notation. In the ASCII character set, the first 32 characters are called control characters. This is why many keyboards have ^ on their control keys. The 26 control codes 1 through 0x1A correspond to ^A through ^Z. PS This relates to the 7-bit ASCII character set, American Standard Code for Information Interchange, composed of control characters for communication control, letters, numbers, and punctuation.
  - deleted 4y ago
    [deleted]
- vlovich123 4y ago
  Usually that’s rendered as one control unit not as caret + H. The caret indicates an escape character. You can typically type in escape sequences via ctrl v (eg ctrl-v + tab for the tab character instead of tab completion)
oars 4y ago
Is the "ctrl + E" shortcut to jump to end of line in a terminal interpreted/executed by the terminal or bash?
- ericbarrett 4y ago
  Bash, via the GNU Readline library.
  - oars 4y ago
    So following the examples in the article, inbetween the sent and recv, it's actually bash interpreting what the sent command is, not the terminal? For example, the ls command example from the article: sent: "l" recv: "l" sent: "s" recv: "s" sent: "\r" recv: "\r\n\x1b[?2004l\r" recv: "file\r\n" recv: "\x1b[?2004hbork@kiwi:/play$ " The interpretation and processing of "\r" to return the final output is actually bash processing this, not the terminal?
    - faho 4y ago
      "\r", "carriage return", is what the return/enter key sends (either that or "\n", it's configurable). So what's being sent from the terminal to bash here is "ls" (which is echoed back) and then the return/enter key, which bash interprets as "run the command". So it sends "\r\n" to the terminal (this is "recv" in that notation), which moves the cursor to the beginning of the line and then to a new line to get the cursor off of the prompt line, and then "\x1b[?2004l", which is the sequence to turn off bracketed paste. Then ls runs and prints "file\r\n", which is the filename "file" on its own line. Then bash takes over again, reenables bracketed paste and prints the prompt. Notably it does not move the cursor to get the prompt on its own line, so when the command didn't end in a newline the prompt hangs in a weird spot - try `printf '%s' foobar`, it'll show your prompt like "foobarbork@kiwi:/play$". There are tricks to get around this.
      - Joker_vD 4y ago
        I think bash actually sends just "\n", the LF-to-CRLF translation is handled in the tty driver (it used to be part of the kernel, but no longer. Funny how Linux still has to translate text to use the so-called "Microsoft line endings" when it comes to terminals).
megous 4y ago
This tells what happens when the virtual terminal receives some input. You can pass that input from a file and it would do the same. There's also a whole lot that happens when you press a key to generate said input (and I don't even mean at the hardware level), which is perhaps much less known.
- deleted 4y ago
  [deleted]
- TacticalCoder 4y ago
  And things turn really "fun" when you begin using modifiers which aren't supported by the USB keyboards specs, like the Hyper key. I've got my Linux / X configured to use Super and Hyper keys but things quickly turn weird. For example the Hyper key works totally fine from Emacs in GUI mode (and, over the years, I assigned a huge number of shortcuts to the Hyper key), since years and years, but I have never spent the time to make it work in the terminal. It's doable but requires some arcane magic. I'm not even talking about having the Hyper key working from a tty (like if I boot in non graphical mode) but simply having it work from, say, an xterm under X. Oh the fun. So until I fix that no "emacs -nw" (emacs in terminal mode) for me as I rely way too much on my Hyper key.
  - aumerle 4y ago
    You dont need to do anything fun. Just convince emacs developers to support a modern terminal keyboard protocol. https://sw.kovidgoyal.net/kitty/keyboard-protocol/ https://sw.kovidgoyal.net/kitty/keyboard-protocol/ and you get support for hyper out of the box.
  - comfypotato 4y ago
    It's more than just the Hyper key (as I'm sure you're aware). The article mentions how C-S-anything is the same as C-anything. I've always wanted to set up a fully functional terminal Emacs (for playing through ssh on a tethered connection in a café) but it seems to be a fairly in-depth process to have an interface with all the things. I use QMK extensively in tandem with my Emacs configuration, and there is a ton of functionality to transpose to the terminal. Ultimately I think I would have to completely redo my configuration in both QMK/KMonad and Emacs to stick to the codes that are sent correctly. Sending "F18 a" with QMK or KMonad (a random prefix-combo I picked for example purposes) instead of "H-whatever" (for whatever keystroke combo you have H-whatever bound) would work with a terminal in your case, but you'd have to change all those bindings and setup QMK/KMonad accordingly. That's altogether too much work.
xwowsersx 4y ago
I really enjoyed reading this.
shadowofneptune 4y ago
Other fun with control characters: The article caught that Alt is ESC, but didn't catch that Ctrl+[ is too. Check it out in a program that uses control codes, like Emacs or Vi. I myself find it somewhat more ergonomic depending on the action.
gumby 4y ago
She wrote that “everyone had one” but really it meant “everyone online at a given time” was using one. In a lot of facilities they were shared.
gumby 4y ago
Look at the ascii chart by bit pattern and the characters will suddenly make sense. Here’s an old chart from the 60s that explains it: https://programesecure.com/ascii-values-table-generator-in-c/ https://programesecure.com/ascii-values-table-generator-in-c... Unfortunately if you just see the character codes with the decimal, hex, and octal next to them this logic is obscured. Remember, it had to be implementable in (mechanical!) hardware.
- zoomablemind 4y ago
  > ...Look at the ascii chart by bit pattern and the characters will suddenly make sense. Could you elaborate on what does the bit pattern reveal? For example, I understand that Ctrl-C generates 0x30, which stands for ETX (end of transmission), but what is it there in 011?
  - aap_ 4y ago
    Ctrl clears the highest two bits. E.g. @ (ascii 0x40) turns into ^@/NUL (ascii 0x00). Similarly C (0x43) -> ^C/ETX (0x03). Older terminals (like the VT05) used bit-pairing also with shift. They just flipped some bits depending on the upper two bits. Compare column 2 with 3 in the linked graphic, and 4,5 with 6,7.
    - gnubison 4y ago
      Closer to toggling in the case of ^?.
      - aap_ 4y ago
        good point!
  - jart 4y ago
    I don't know if the chart helps, but #define CTRL(x) ((x) ^ 0100) // e.g. 'C' ^ 0100 is 3 // e.g. '@' ^ 0100 is 0 // e.g. 'M' ^ 0100 is 13 a.k.a. \r a.k.a. enter I like this explanation better. It also explains what notation like ^C means. It's shorthand for 0100^C.
raggi 4y ago
There's a command for doing the diagnosis goterm was used for here, it's called script(1).
niels_bom 4y ago
Julia Evans is such a treasure. I’ve learned a tonne reading her stuff.
schoen 4y ago
> I believe the reason cat gets interrupted when we press Ctrl+C is that the Linux kernel on the server side receives this \x03 character, recognizes that it means “interrupt”, and then sends a SIGINT to the process that owns the pseudoterminal’s process group. So it’s handled in the kernel and not in userspace. Also interestingly, the use of \x03 for this purpose is a default, but it's not hardcoded. You can change it with the stty command. For example, if you run stty intr '^X' then your interrupt character sequence will be Ctrl+X instead of Ctrl+C (!). In order to make this change, the stty program actually has to call into the kernel with an ioctl call (TCSETS or a related ioctl, for "terminal control set settings"). You can learn more about this ioctl on Linux in the ioctl_tty(2) man page, and you can see the various settings that can be changed this way with stty -a (It's a little bit confusing which things are handled by the readline library and which things are handled by the kernel, as the kernel's ability to support some rudimentary line-editing features on an interactive terminal long predates libraries like readline. I think readline might actually disable kernel interpretation of some of these control characters when it starts accepting input, and then re-enable them when it stops, but I've never looked into that.)
- formerly_proven 4y ago
  > I think readline might actually disable kernel interpretation of some of these control characters when it starts accepting input, and then re-enable them when it stops, but I've never looked into that. That's the canonical mode bit (ICANON). Canonical mode means the kernel/terminal is line-buffered and line-editing is handled by it; non-canonical mode means user input is pushed to the application immediately.
- tonnydourado 4y ago
  > For example, if you run > stty intr '^X' > then your interrupt character sequence will be Ctrl+X instead of Ctrl+C (!). I have a feeling that if you actually do this in a real Linux system, a sysadmin will hunt you down and dismember you alive, but I might be wrong.
  - deadbunny 4y ago
    Can confirm.
noncoml 4y ago
2004h -> “Turn on bracketed paste mode” 2004l -> “Turn off bracketed paste mode” https://en.m.wikipedia.org/wiki/ANSI_escape_code#CSIsection https://en.m.wikipedia.org/wiki/ANSI_escape_code#CSIsection
- durazabu 4y ago
  Same mechanism that we could use to disambiguate keys like ^H and backspace. http://www.leonerd.org.uk/hacks/fixterms/ http://www.leonerd.org.uk/hacks/fixterms/
  - aumerle 4y ago
    That is obsolete and wrong in many ways: https://sw.kovidgoyal.net/kitty/keyboard-protocol/#bugs-in-fixterms https://sw.kovidgoyal.net/kitty/keyboard-protocol/#bugs-in-f...
    - kps 4y ago
      Not to mention that it was NIH after xterm already had such a mode.
    - durazabu 4y ago
      Fascinating, thank you for informing me.
benreesman 4y ago
I don’t know Julia Evans, certainly not her academic background. But she blogs eloquently from the standpoint of someone learning things from first principles, in an accessible, actionable way. As a self-taught person this is near and dear to me. She’s like the 3Blue1Brown of systems programming, and there’s not much higher praise. Striking the balance between rigor and practical applicability is tough, especially while assuming little prior knowledge. Keep up the good work @jvns!
- zero_shift 4y ago
  She was a Staff Engineer at Stripe. She wrote a great primer on SQL too. https://jvns.ca/ https://jvns.ca/ is a treasure trove
  - benreesman 4y ago
    Stripe is an interesting place. It’s clearly a great business, but they’ve let that bleed over a bit into thinking their p95 people are Google p95, which is trivially silly. It’s an awesome company, and they’ve got solid folks, but “Staff” at Stripe isn’t what makes Julia cool: Julia’s work with Recurse is way cooler. I mean we’ve got Consul: https://stripe.com/blog/service-discovery-at-stripe https://stripe.com/blog/service-discovery-at-stripe And then etcd: https://stripe.com/blog/operating-kubernetes https://stripe.com/blog/operating-kubernetes Both courtesy of Julia incidentally. Chubby has worked since like 2003. We’re just talking a different level of ball game.
mongol 4y ago
Have anyone attempted to connect a serial terminal over bluetooth? At some point I thought that could be a neat idea because it would be "internet secure". I don't remember the circumstances exactly, but I never got it to work fully, there were some echoing of characters that I never figured out how to configure away.
Wildgoose 4y ago
The point of echoing back what was received is not because the client is "really dumb", it's actually good engineering practise. It directly confirms to the user what the machine believes it has received. If this is incorrect, (whether caused by line noise or whatever), then the user knows this immediately. Unix has a lot of 2-character commands - you wouldn't want an innocent command accidentally mistranslated to "rm" and then press <Return>.
- rmetzler 4y ago
  Yes, state is only modified on the server, not in the dumb clients. Weakening this fundamental idea is what makes it hard to write modern single page applications.
- grayfaced 4y ago
  It also means the server can turn off echo for sensitive things like passwords.
  - gnubison 4y ago
    And, more importantly, for programs like vi and Emacs that want to handle rendering themselves.
jart 4y ago
> remote terminals are very old technology TTYs or teletypewriters have been in use since the 19th century. I'd love to see a blog post that talks more about the early history.
pphysch 4y ago
It's a valuable exercise to write a web shell service with Go standard library (rather than goterm). It shouldn't take more than 100-200 LOC and you will learn a lot about how SSH protocol, streaming IO, websockets work.
- SheetPost 4y ago
  ah, it's also a great deal of a distraction if your goal was to see what gets sent when presing keys
  - pphysch 4y ago
    On the contrary, you don't have to do much more work than the author already did, and you get deep practical insights into how this very useful technology works beyond the marginally useful trivia of how terminals happen to internally represent keystrokes
- zero_shift 4y ago
  There should be a body of these "koans", little program specs to implement to explore various technologies.
prosaic-hacker 4y ago
It is amazing how the understanding of what happens in our computers goes from : the air we breath to a mystery. Although if you asked one of my 900 coworkers in 1981 who wrote flight simulation software in Fortran, how the VT100 terminals interacted with the computers they were connected to,only a few dozen would know. One of my projects at the time was to create a library to support the creation of 'screen oriented' applications without knowing the escape sequences.
amelius 4y ago
For graphical interfaces we went from X11 to Wayland. Is anybody working on a replacement technology for text interfaces?
- drainyard 4y ago
  Something like Terminal Click? https://media.handmade-seattle.com/terminal-click/ https://media.handmade-seattle.com/terminal-click/
- atq2119 4y ago
  Why? I don't think there is a clear and compelling case for doing this given the ecosystem issues. Meanwhile, there's slow but interesting innovation happening in this space e.g. around kitty.
kristopolous 4y ago
I was imagining this would be an overview of the circuit closing upon the key press and the chips that translate it to the signal that traverses to the the USB connector and goes through a controller chip etc... From the mechanical closing of the circuit to the illumination at the display. Maybe I should give that a go in some classic Tracy Kidder style. I certainly can't fill in all those steps. I'd have to do some learning myself
- deleted 4y ago
  [deleted]
- branchcontrol 4y ago
  A slightly different exploration: https://youtu.be/XUdxXON27xA https://youtu.be/XUdxXON27xA
- joezydeco 4y ago
  It's actually a useful interview question in some positions, like embedded.
  - kristopolous 4y ago
    I've gotten it before and I had no idea <how> to answer. This is characteristic of one of the common interview traps I fall in. I don't know who my audience is or what kind of answer they're looking for. I can answer the software world version of that all day along with internationalization and the history. But hardware? No not really
lqet 4y ago
> The client sends l and then immediately receives an l sent back. I guess the idea here is that the client is really dumb – it doesn’t know that when I type an l, I want an l to be echoed back to the screen. It has to be told explicitly by the server process to display it. Of course the client may be really dumb! Here is a 1930s teletype used as a Linux terminal: https://youtu.be/2XLZ4Z8LpEE?t=656 https://youtu.be/2XLZ4Z8LpEE?t=656 If you would not do it that way, you would run into all kinds of synchronicity problems. How could you be sure that, after rapidly making 50 keystrokes, every keystroke was received by the other end, in that order?
- freedomben 4y ago
  > How could you be sure that, after rapidly making 50 keystrokes, every keystroke was received by the other end, in that order? Exactly. I implemented a client that emulated a VT100 early in my career, and this is a real problem. There are various strategies you can use but by far the simplest and safest seemed to just be the echo and for the client to always display exactly what it receives[1]. There's nothing worse than typing out a command that you realize is wrong and potentially destructive, only to Ctrl+U it and have the client kill the line but the server didn't get the instruction, so when you press enter it runs the evil command. If the command doesn't echo anything you may not even know! I once accidented a space in the path I was deleting when (recklessly with -rf) trying to remove my ~/bin directory, like this: rm -rf ~ /bin Good Lord that was a bad day. Thankfully I still had the installation disc to restore /bin, and a relatively recent backup of my home directory to restore that. I lost a few days of uncommitted code, but that felt like a trifle compared to what it could have been :-) [1]: I love how mosh[2] handles this to get the best of both worlds. It will smartly show you what you typed, but sill underline it until it actually receives the echo from the server, so you can type and feel like there's no delay between bytes, but still be confident that client state matches the server state. [2]: https://mosh.org/ https://mosh.org/
- icedchai 4y ago
  This was definitely a problem in the dialup modem days before error correction, where line noise might add extra characters, or modify what you were sending.
- 4y ago
smelbe 4y ago
And this, guys, is the reason why it was able to do fun things to other people's computers when their operating system did not inform the modem to cease receiving incoming bytes when the line was dropped and then picked back up again. It did not care where the bytes came from; it just accepted them...
mignato 4y ago
- shmde 4y ago
  @dang can you ban this spam/bot account.
mignato 4y ago
[flagged]
Linda703 4y ago
[dead]
deleted 4y ago
[deleted]
jstimpfle 4y ago
I think the canonical resource on this is https://www.linusakesson.net/programming/tty/ https://www.linusakesson.net/programming/tty/
pjmlp 4y ago
This is what happens on a classical UNIX terminal, other platforms have different workflows.
- mega_dingus 4y ago
  e.g. VMS would not echo back keystrokes until it was truly ready to accept input No typeahead while a previous command is running, like you can do in UNIX et al
  - kps 4y ago
    And many mainframe (IBM et al) terminals worked more like HTML forms — typing modified only the local screen, with a key to send the entire current state to the host.
whalesalad 4y ago
As a kid my mom would take my sister and I to the big Pasadena Public Library which had these DEC terminals all over the place for using the card catalog to find books. I was fascinated by them and would often just sit and play hacker. Might even be a big reason for why I am a programmer today.
bryanrasmussen 4y ago
I might be really tempted to say "my terminal doesn't have any keys" if this came up in an interview.
throw0101a 4y ago
Nothing about scancodes and mapping the physical location of the key to the character depending on the keyboard layout (QWERTY/AZERTY/etc)? * https://en.wikipedia.org/wiki/Scancode https://en.wikipedia.org/wiki/Scancode * https://en.wikipedia.org/wiki/Keyboard_layout https://en.wikipedia.org/wiki/Keyboard_layout
sunyc 4y ago
Next step is to explain how ctrl+c work over ssh , it is my favorite system engineering interview question.
- Joker_vD 4y ago
  The ssh client on your local end receives SIGINT and processes it by sending a special kind of packet over the ssh session to the other side; the sshd on the remote side receives this special packet and processes it by sending SIGINT to whatever command it has originally spawned. IIRC telnet instead uses urgent TCP packets to indicate SIGINT.
  - gnubison 4y ago
    My guess: The ssh client on your local end receives a 0x03 byte directly from the terminal, because it has disabled ICANON and ISIG and whatnot, and forwards it to the remote connection. The remote sshd then feeds the 0x03 byte to the pseudoterminal it has setup, and then the (remote) kernel may or may not interpret that as SIGINT. For example, ISIG could be disabled (“stty -isig”) or it might be a different key (e.g. “stty intr '^X'”, as mentioned elsewhere in this thread). Your setup fails to distinguish keyboard interrupts (intended for the remote machine) and real SIGINTs generated by kill(1). It also uses the local termios(4) settings instead of the remote ones.
    - Joker_vD 4y ago
      You may actually be right; I know that SSH channel protocol has special message kind specifically for sending signals to the remote process, which is different from channel messages with normal data, but I don't know if it's used by actual ssh client implementations. They may simply just put local tty in raw mode and forward all input from it as normal data. Hard to say without looking at the actual code, and right now I am not particularly in a mood for reading C sources at the moment. Maybe someone else is and will tell us the true story! P.S. Now that I think of it, ssh implementations have to "sync" local and remote tty parameters or at least make it look sane for the user: if you resize your local xterm, arguably the remote e.g. vi should get notified, but what if it's the remote process that changes the terminal dimensions, should your local xterm get resized as a result?
tpoacher 4y ago
This also partly explains why a lot of keystrokes in nano do the same thing and cannot be bound separately (see attached file keystrokes.nanorc in https://savannah.gnu.org/bugs/index.php?61699 https://savannah.gnu.org/bugs/index.php?61699 )
evernerdy 4y ago
Love this! The format for some reason made me flash back to using Telnet for IRC. lol

408 points by robenkleene 4y ago

The picture of the VT100 brings back memories. I had one at home and would connect to our Vax 11/780 to program in FORTRAN using the EDT editor... using a 1200 baud modem. (about 1981 or so)

bjelkeman-again 4y ago
I wish I had kept my amber terminal. Great for working late at night, very focused. Having a modern display projecting amber colours just isn’t the same.
lambic 4y ago
I shared a flat in London with two other geeks in the 90s and we wired up vt100 terminals in each of our rooms so we could talk to each other without leaving our rooms. They didn't last long, they were quickly replaced by PCs on a lan so we could play Doom.

andreskytt 4y ago

And this, kids, is why it was possible to do fun things to computers of other people when their os did not tell the modem to stop accepting incoming bytes when the line was dropped and then picked up again. It just accepted bytes from wherever they might come…

oars 4y ago

So I didn't know that when I press backspace in the terminal, "x08" is being sent, not "^H". What is the purpose of caret notation then? Is it just for human readability? e.g. my terminal shows ^H sometimes when I press backspace

yjftsjthsd-h 4y ago
Pretty sure it's for the human, yes; how else would you represent "hey human, something just echoed the thing you get from ctrl-h"? (Keeping in mind that this almost certainly predates unicode and fancy fonts, and probably colors in the terminal. I'm open to the idea that we could do better today, but it's hard to overcome 50 years of tradition.)
IncRnd 4y ago
> So I didn't know that when I press backspace in the terminal, "x08" is being sent, not "^H". "0x08" is "^H". H is the eighth letter of the alphabet. There is no difference between those except for the notation. In the ASCII character set, the first 32 characters are called control characters. This is why many keyboards have ^ on their control keys. The 26 control codes 1 through 0x1A correspond to ^A through ^Z. PS This relates to the 7-bit ASCII character set, American Standard Code for Information Interchange, composed of control characters for communication control, letters, numbers, and punctuation.
- deleted 4y ago
  [deleted]
vlovich123 4y ago
Usually that’s rendered as one control unit not as caret + H. The caret indicates an escape character. You can typically type in escape sequences via ctrl v (eg ctrl-v + tab for the tab character instead of tab completion)

oars 4y ago

Is the "ctrl + E" shortcut to jump to end of line in a terminal interpreted/executed by the terminal or bash?

ericbarrett 4y ago
Bash, via the GNU Readline library.
- oars 4y ago
  So following the examples in the article, inbetween the sent and recv, it's actually bash interpreting what the sent command is, not the terminal? For example, the ls command example from the article: sent: "l" recv: "l" sent: "s" recv: "s" sent: "\r" recv: "\r\n\x1b[?2004l\r" recv: "file\r\n" recv: "\x1b[?2004hbork@kiwi:/play$ " The interpretation and processing of "\r" to return the final output is actually bash processing this, not the terminal?
  - faho 4y ago
    "\r", "carriage return", is what the return/enter key sends (either that or "\n", it's configurable). So what's being sent from the terminal to bash here is "ls" (which is echoed back) and then the return/enter key, which bash interprets as "run the command". So it sends "\r\n" to the terminal (this is "recv" in that notation), which moves the cursor to the beginning of the line and then to a new line to get the cursor off of the prompt line, and then "\x1b[?2004l", which is the sequence to turn off bracketed paste. Then ls runs and prints "file\r\n", which is the filename "file" on its own line. Then bash takes over again, reenables bracketed paste and prints the prompt. Notably it does not move the cursor to get the prompt on its own line, so when the command didn't end in a newline the prompt hangs in a weird spot - try `printf '%s' foobar`, it'll show your prompt like "foobarbork@kiwi:/play$". There are tricks to get around this.
    - Joker_vD 4y ago
      I think bash actually sends just "\n", the LF-to-CRLF translation is handled in the tty driver (it used to be part of the kernel, but no longer. Funny how Linux still has to translate text to use the so-called "Microsoft line endings" when it comes to terminals).

megous 4y ago

This tells what happens when the virtual terminal receives some input. You can pass that input from a file and it would do the same. There's also a whole lot that happens when you press a key to generate said input (and I don't even mean at the hardware level), which is perhaps much less known.

deleted 4y ago
[deleted]
TacticalCoder 4y ago
And things turn really "fun" when you begin using modifiers which aren't supported by the USB keyboards specs, like the Hyper key. I've got my Linux / X configured to use Super and Hyper keys but things quickly turn weird. For example the Hyper key works totally fine from Emacs in GUI mode (and, over the years, I assigned a huge number of shortcuts to the Hyper key), since years and years, but I have never spent the time to make it work in the terminal. It's doable but requires some arcane magic. I'm not even talking about having the Hyper key working from a tty (like if I boot in non graphical mode) but simply having it work from, say, an xterm under X. Oh the fun. So until I fix that no "emacs -nw" (emacs in terminal mode) for me as I rely way too much on my Hyper key.
- aumerle 4y ago
  You dont need to do anything fun. Just convince emacs developers to support a modern terminal keyboard protocol. https://sw.kovidgoyal.net/kitty/keyboard-protocol/ https://sw.kovidgoyal.net/kitty/keyboard-protocol/ and you get support for hyper out of the box.
- comfypotato 4y ago
  It's more than just the Hyper key (as I'm sure you're aware). The article mentions how C-S-anything is the same as C-anything. I've always wanted to set up a fully functional terminal Emacs (for playing through ssh on a tethered connection in a café) but it seems to be a fairly in-depth process to have an interface with all the things. I use QMK extensively in tandem with my Emacs configuration, and there is a ton of functionality to transpose to the terminal. Ultimately I think I would have to completely redo my configuration in both QMK/KMonad and Emacs to stick to the codes that are sent correctly. Sending "F18 a" with QMK or KMonad (a random prefix-combo I picked for example purposes) instead of "H-whatever" (for whatever keystroke combo you have H-whatever bound) would work with a terminal in your case, but you'd have to change all those bindings and setup QMK/KMonad accordingly. That's altogether too much work.

xwowsersx 4y ago

I really enjoyed reading this.

shadowofneptune 4y ago

Other fun with control characters: The article caught that Alt is ESC, but didn't catch that Ctrl+[ is too. Check it out in a program that uses control codes, like Emacs or Vi. I myself find it somewhat more ergonomic depending on the action.

gumby 4y ago

She wrote that “everyone had one” but really it meant “everyone online at a given time” was using one. In a lot of facilities they were shared.

gumby 4y ago

Look at the ascii chart by bit pattern and the characters will suddenly make sense. Here’s an old chart from the 60s that explains it: https://programesecure.com/ascii-values-table-generator-in-c/ https://programesecure.com/ascii-values-table-generator-in-c... Unfortunately if you just see the character codes with the decimal, hex, and octal next to them this logic is obscured. Remember, it had to be implementable in (mechanical!) hardware.

zoomablemind 4y ago
> ...Look at the ascii chart by bit pattern and the characters will suddenly make sense. Could you elaborate on what does the bit pattern reveal? For example, I understand that Ctrl-C generates 0x30, which stands for ETX (end of transmission), but what is it there in 011?
- aap_ 4y ago
  Ctrl clears the highest two bits. E.g. @ (ascii 0x40) turns into ^@/NUL (ascii 0x00). Similarly C (0x43) -> ^C/ETX (0x03). Older terminals (like the VT05) used bit-pairing also with shift. They just flipped some bits depending on the upper two bits. Compare column 2 with 3 in the linked graphic, and 4,5 with 6,7.
  - gnubison 4y ago
    Closer to toggling in the case of ^?.
    - aap_ 4y ago
      good point!
- jart 4y ago
  I don't know if the chart helps, but #define CTRL(x) ((x) ^ 0100) // e.g. 'C' ^ 0100 is 3 // e.g. '@' ^ 0100 is 0 // e.g. 'M' ^ 0100 is 13 a.k.a. \r a.k.a. enter I like this explanation better. It also explains what notation like ^C means. It's shorthand for 0100^C.

raggi 4y ago

There's a command for doing the diagnosis goterm was used for here, it's called script(1).

niels_bom 4y ago

Julia Evans is such a treasure. I’ve learned a tonne reading her stuff.

schoen 4y ago

> I believe the reason cat gets interrupted when we press Ctrl+C is that the Linux kernel on the server side receives this \x03 character, recognizes that it means “interrupt”, and then sends a SIGINT to the process that owns the pseudoterminal’s process group. So it’s handled in the kernel and not in userspace. Also interestingly, the use of \x03 for this purpose is a default, but it's not hardcoded. You can change it with the stty command. For example, if you run stty intr '^X' then your interrupt character sequence will be Ctrl+X instead of Ctrl+C (!). In order to make this change, the stty program actually has to call into the kernel with an ioctl call (TCSETS or a related ioctl, for "terminal control set settings"). You can learn more about this ioctl on Linux in the ioctl_tty(2) man page, and you can see the various settings that can be changed this way with stty -a (It's a little bit confusing which things are handled by the readline library and which things are handled by the kernel, as the kernel's ability to support some rudimentary line-editing features on an interactive terminal long predates libraries like readline. I think readline might actually disable kernel interpretation of some of these control characters when it starts accepting input, and then re-enable them when it stops, but I've never looked into that.)

formerly_proven 4y ago
> I think readline might actually disable kernel interpretation of some of these control characters when it starts accepting input, and then re-enable them when it stops, but I've never looked into that. That's the canonical mode bit (ICANON). Canonical mode means the kernel/terminal is line-buffered and line-editing is handled by it; non-canonical mode means user input is pushed to the application immediately.
tonnydourado 4y ago
> For example, if you run > stty intr '^X' > then your interrupt character sequence will be Ctrl+X instead of Ctrl+C (!). I have a feeling that if you actually do this in a real Linux system, a sysadmin will hunt you down and dismember you alive, but I might be wrong.
- deadbunny 4y ago
  Can confirm.

noncoml 4y ago

2004h -> “Turn on bracketed paste mode” 2004l -> “Turn off bracketed paste mode” https://en.m.wikipedia.org/wiki/ANSI_escape_code#CSIsection https://en.m.wikipedia.org/wiki/ANSI_escape_code#CSIsection

durazabu 4y ago
Same mechanism that we could use to disambiguate keys like ^H and backspace. http://www.leonerd.org.uk/hacks/fixterms/ http://www.leonerd.org.uk/hacks/fixterms/
- aumerle 4y ago
  That is obsolete and wrong in many ways: https://sw.kovidgoyal.net/kitty/keyboard-protocol/#bugs-in-fixterms https://sw.kovidgoyal.net/kitty/keyboard-protocol/#bugs-in-f...
  - kps 4y ago
    Not to mention that it was NIH after xterm already had such a mode.
  - durazabu 4y ago
    Fascinating, thank you for informing me.

benreesman 4y ago

I don’t know Julia Evans, certainly not her academic background. But she blogs eloquently from the standpoint of someone learning things from first principles, in an accessible, actionable way. As a self-taught person this is near and dear to me. She’s like the 3Blue1Brown of systems programming, and there’s not much higher praise. Striking the balance between rigor and practical applicability is tough, especially while assuming little prior knowledge. Keep up the good work @jvns!

zero_shift 4y ago
She was a Staff Engineer at Stripe. She wrote a great primer on SQL too. https://jvns.ca/ https://jvns.ca/ is a treasure trove
- benreesman 4y ago
  Stripe is an interesting place. It’s clearly a great business, but they’ve let that bleed over a bit into thinking their p95 people are Google p95, which is trivially silly. It’s an awesome company, and they’ve got solid folks, but “Staff” at Stripe isn’t what makes Julia cool: Julia’s work with Recurse is way cooler. I mean we’ve got Consul: https://stripe.com/blog/service-discovery-at-stripe https://stripe.com/blog/service-discovery-at-stripe And then etcd: https://stripe.com/blog/operating-kubernetes https://stripe.com/blog/operating-kubernetes Both courtesy of Julia incidentally. Chubby has worked since like 2003. We’re just talking a different level of ball game.

mongol 4y ago

Have anyone attempted to connect a serial terminal over bluetooth? At some point I thought that could be a neat idea because it would be "internet secure". I don't remember the circumstances exactly, but I never got it to work fully, there were some echoing of characters that I never figured out how to configure away.

Wildgoose 4y ago

The point of echoing back what was received is not because the client is "really dumb", it's actually good engineering practise. It directly confirms to the user what the machine believes it has received. If this is incorrect, (whether caused by line noise or whatever), then the user knows this immediately. Unix has a lot of 2-character commands - you wouldn't want an innocent command accidentally mistranslated to "rm" and then press <Return>.

rmetzler 4y ago
Yes, state is only modified on the server, not in the dumb clients. Weakening this fundamental idea is what makes it hard to write modern single page applications.
grayfaced 4y ago
It also means the server can turn off echo for sensitive things like passwords.
- gnubison 4y ago
  And, more importantly, for programs like vi and Emacs that want to handle rendering themselves.

jart 4y ago

> remote terminals are very old technology TTYs or teletypewriters have been in use since the 19th century. I'd love to see a blog post that talks more about the early history.

pphysch 4y ago

It's a valuable exercise to write a web shell service with Go standard library (rather than goterm). It shouldn't take more than 100-200 LOC and you will learn a lot about how SSH protocol, streaming IO, websockets work.

SheetPost 4y ago
ah, it's also a great deal of a distraction if your goal was to see what gets sent when presing keys
- pphysch 4y ago
  On the contrary, you don't have to do much more work than the author already did, and you get deep practical insights into how this very useful technology works beyond the marginally useful trivia of how terminals happen to internally represent keystrokes
zero_shift 4y ago
There should be a body of these "koans", little program specs to implement to explore various technologies.

prosaic-hacker 4y ago

It is amazing how the understanding of what happens in our computers goes from : the air we breath to a mystery. Although if you asked one of my 900 coworkers in 1981 who wrote flight simulation software in Fortran, how the VT100 terminals interacted with the computers they were connected to,only a few dozen would know. One of my projects at the time was to create a library to support the creation of 'screen oriented' applications without knowing the escape sequences.

amelius 4y ago

For graphical interfaces we went from X11 to Wayland. Is anybody working on a replacement technology for text interfaces?

drainyard 4y ago
Something like Terminal Click? https://media.handmade-seattle.com/terminal-click/ https://media.handmade-seattle.com/terminal-click/
atq2119 4y ago
Why? I don't think there is a clear and compelling case for doing this given the ecosystem issues. Meanwhile, there's slow but interesting innovation happening in this space e.g. around kitty.

kristopolous 4y ago

I was imagining this would be an overview of the circuit closing upon the key press and the chips that translate it to the signal that traverses to the the USB connector and goes through a controller chip etc... From the mechanical closing of the circuit to the illumination at the display. Maybe I should give that a go in some classic Tracy Kidder style. I certainly can't fill in all those steps. I'd have to do some learning myself

deleted 4y ago
[deleted]
branchcontrol 4y ago
A slightly different exploration: https://youtu.be/XUdxXON27xA https://youtu.be/XUdxXON27xA
joezydeco 4y ago
It's actually a useful interview question in some positions, like embedded.
- kristopolous 4y ago
  I've gotten it before and I had no idea <how> to answer. This is characteristic of one of the common interview traps I fall in. I don't know who my audience is or what kind of answer they're looking for. I can answer the software world version of that all day along with internationalization and the history. But hardware? No not really

lqet 4y ago

> The client sends l and then immediately receives an l sent back. I guess the idea here is that the client is really dumb – it doesn’t know that when I type an l, I want an l to be echoed back to the screen. It has to be told explicitly by the server process to display it. Of course the client may be really dumb! Here is a 1930s teletype used as a Linux terminal: https://youtu.be/2XLZ4Z8LpEE?t=656 https://youtu.be/2XLZ4Z8LpEE?t=656 If you would not do it that way, you would run into all kinds of synchronicity problems. How could you be sure that, after rapidly making 50 keystrokes, every keystroke was received by the other end, in that order?

freedomben 4y ago
> How could you be sure that, after rapidly making 50 keystrokes, every keystroke was received by the other end, in that order? Exactly. I implemented a client that emulated a VT100 early in my career, and this is a real problem. There are various strategies you can use but by far the simplest and safest seemed to just be the echo and for the client to always display exactly what it receives[1]. There's nothing worse than typing out a command that you realize is wrong and potentially destructive, only to Ctrl+U it and have the client kill the line but the server didn't get the instruction, so when you press enter it runs the evil command. If the command doesn't echo anything you may not even know! I once accidented a space in the path I was deleting when (recklessly with -rf) trying to remove my ~/bin directory, like this: rm -rf ~ /bin Good Lord that was a bad day. Thankfully I still had the installation disc to restore /bin, and a relatively recent backup of my home directory to restore that. I lost a few days of uncommitted code, but that felt like a trifle compared to what it could have been :-) [1]: I love how mosh[2] handles this to get the best of both worlds. It will smartly show you what you typed, but sill underline it until it actually receives the echo from the server, so you can type and feel like there's no delay between bytes, but still be confident that client state matches the server state. [2]: https://mosh.org/ https://mosh.org/
icedchai 4y ago
This was definitely a problem in the dialup modem days before error correction, where line noise might add extra characters, or modify what you were sending.
4y ago

smelbe 4y ago

And this, guys, is the reason why it was able to do fun things to other people's computers when their operating system did not inform the modem to cease receiving incoming bytes when the line was dropped and then picked back up again. It did not care where the bytes came from; it just accepted them...

mignato 4y ago

shmde 4y ago
@dang can you ban this spam/bot account.

mignato 4y ago

[flagged]

Linda703 4y ago

[dead]

deleted 4y ago

[deleted]

jstimpfle 4y ago

I think the canonical resource on this is https://www.linusakesson.net/programming/tty/ https://www.linusakesson.net/programming/tty/

pjmlp 4y ago

This is what happens on a classical UNIX terminal, other platforms have different workflows.

mega_dingus 4y ago
e.g. VMS would not echo back keystrokes until it was truly ready to accept input No typeahead while a previous command is running, like you can do in UNIX et al
- kps 4y ago
  And many mainframe (IBM et al) terminals worked more like HTML forms — typing modified only the local screen, with a key to send the entire current state to the host.

whalesalad 4y ago

As a kid my mom would take my sister and I to the big Pasadena Public Library which had these DEC terminals all over the place for using the card catalog to find books. I was fascinated by them and would often just sit and play hacker. Might even be a big reason for why I am a programmer today.

bryanrasmussen 4y ago

I might be really tempted to say "my terminal doesn't have any keys" if this came up in an interview.

throw0101a 4y ago

Nothing about scancodes and mapping the physical location of the key to the character depending on the keyboard layout (QWERTY/AZERTY/etc)? * https://en.wikipedia.org/wiki/Scancode https://en.wikipedia.org/wiki/Scancode * https://en.wikipedia.org/wiki/Keyboard_layout https://en.wikipedia.org/wiki/Keyboard_layout

sunyc 4y ago

Next step is to explain how ctrl+c work over ssh , it is my favorite system engineering interview question.

Joker_vD 4y ago
The ssh client on your local end receives SIGINT and processes it by sending a special kind of packet over the ssh session to the other side; the sshd on the remote side receives this special packet and processes it by sending SIGINT to whatever command it has originally spawned. IIRC telnet instead uses urgent TCP packets to indicate SIGINT.
- gnubison 4y ago
  My guess: The ssh client on your local end receives a 0x03 byte directly from the terminal, because it has disabled ICANON and ISIG and whatnot, and forwards it to the remote connection. The remote sshd then feeds the 0x03 byte to the pseudoterminal it has setup, and then the (remote) kernel may or may not interpret that as SIGINT. For example, ISIG could be disabled (“stty -isig”) or it might be a different key (e.g. “stty intr '^X'”, as mentioned elsewhere in this thread). Your setup fails to distinguish keyboard interrupts (intended for the remote machine) and real SIGINTs generated by kill(1). It also uses the local termios(4) settings instead of the remote ones.
  - Joker_vD 4y ago
    You may actually be right; I know that SSH channel protocol has special message kind specifically for sending signals to the remote process, which is different from channel messages with normal data, but I don't know if it's used by actual ssh client implementations. They may simply just put local tty in raw mode and forward all input from it as normal data. Hard to say without looking at the actual code, and right now I am not particularly in a mood for reading C sources at the moment. Maybe someone else is and will tell us the true story! P.S. Now that I think of it, ssh implementations have to "sync" local and remote tty parameters or at least make it look sane for the user: if you resize your local xterm, arguably the remote e.g. vi should get notified, but what if it's the remote process that changes the terminal dimensions, should your local xterm get resized as a result?

tpoacher 4y ago

This also partly explains why a lot of keystrokes in nano do the same thing and cannot be bound separately (see attached file keystrokes.nanorc in https://savannah.gnu.org/bugs/index.php?61699 https://savannah.gnu.org/bugs/index.php?61699 )

evernerdy 4y ago

Love this! The format for some reason made me flash back to using Telnet for IRC. lol