5 ms·

What would a clean-slate implementation of a serial terminal look like today? I work in embedded, and a serial terminal is practically a must in any system, if

What would a clean-slate implementation of a serial terminal look like today? I work in embedded, and a serial terminal is practically a must in any system, if only for debug. A way to provide input and receive output. You'll definitely want a FIFOs for RX/TX. You also likely want control chars and a way to shortcut the FIFO, if only to interrupt output. The FIFO means you can abstract away the exact physical communication medium. TUI interfaces mean you also want a way to tell the program what the dimensions of your screen are, through the control chars. What else?

R0b0t1 6y ago
I've given this some thought. The way to improve the TTY is to give it channels, like SSH or RDP. This way you can transit unrelated data over the same connection. And, if you think about it, most of the complexity of a serial interface is handling mode settings and state; streams abstract that away for you. Streams get rid of the need for control characters and in band signalling. Streams could be given priority and interrupt levels.
- ksherlock 6y ago
  If you add a header with the channel number and data length and a CRC, it starts looking like UDP.
- tyingq 6y ago
  RS-485 is essentially RS-232 with channels. Never used it, but Linux supports it at some level.
  - nrclark 6y ago
    This might not be accurate. RS-485 is an electrical signaling standard for carrying arbitrary binary signals. It's usually used for carrying UART data, as a longer-range / higher-speed replacement for RS-232. It has no opinion about what is transmitted, and has no concept of channels or even any particular encoding format.
    - tyingq 6y ago
      It could be the basis for a tty with multiple channels, fairly easily.
      - catblast01 6y ago
        How does that make it an RS-232 with channels? You’re confused. RS-485 just like RS-232 is an electrical specification for serial signaling (with multiple access), nothing more nor less. There are USB to 485 converters that basically operate exactly the same as their serial port counterparts and will show up as a tty device in Linux/Unix flavor or a COM port in Windows.
        tyingq 6y ago
        It's a bus, rather than point-to-point. Schemes to address devices on the bus separately exist. You could call them "channels" even. RS-232 can't do that, as it isn't an electrical bus. Respectfully, you're confused. Since tty's are fairly well married to a serial port, adding an ioctl() (or similar) that would switch channels would seem likely. RS-485 would provide a way for that to work not just with ptys, but with real physical devices as well. With reasonable backward compatibility.
        catblast01 6y ago
        This is really reaching. Channelization as being referred to upthread in reference to SSH or RDP is a protocol level abstraction, rs485 bussing is at the electrical level for multiple physical devices and also doesn’t specify any addressing scheme, that would be specified as a higher level protocol (that could be done just as well over rs-232). Device addressing and channelization are not even logically the same thing. Ethernet as it is typically deployed is not a bus either but it is used in channelized communication (packet switched no less) just fine. Edit: responding to your edit. The point still stands this is orthogonal to the physical or electrical layer. The vast vast majority of the internet including Ethernet and fiber links are point to point and clearly they carry a tremendous number of separate streams, and a small amount of broadcast. You can run layer 2 over an RS-232C serial port, whether its Ethernet, PPP, SLIP or ATM. In Linux PPP is in fact implemented as a tty line discipline. As for RS-485, there are widely used standards such as DMX512. It is not a strongly supported idea to put this stuff in the kernel, when performance in user space is sufficient. Finally, RS-232 can do the same thing (multidrop) with electrical repeaters. See MAX3322 eg.
  - kyberias 6y ago
    RS-485 in itself really isn't. It has no concept of "channels" although it can be operated as multipoint. You may be mixing this with Modbus which is typically run on RS-485.
- gonzus 6y ago
  A big fat YES from me. I have been looking into TUIs lately, and the state of terminal handling is ridiculously bogged down by years of cruft that, honestly, I don't give a shit about anymore. To put it succinctly: I want Alt-Home and Ctrl-Click to work on a TUI. Period. If I had superpowers, I would layer the system as a very low-level library that recognizes keyboard / mouse / tablet / microphone input hardware events and encodes them somehow (so Alt is a separate key, pressing multiple keys is handled, clicking + moving the mouse is handled, etc); then a protocol that can give you this event stream, either from a local or remote source; then a layer that maps this into higher-level events (so Alt and Home pressed together is interpreted as Alt-Home). The idea would be to have the ability to detect, even over a serial connection, any of these input events, without resorting to escape codes. I don't even know if this all makes sense; I just hate the loops you have to go through today to even try to make sense of key combinations. /rant
kragen 6y ago
Are you talking about a physical machine or a protocol design? I don't think you can abstract away the exact physical communication medium from a physical machine (or your cable won't fit in the hole, or it'll burn out your circuitry), and the protocol design probably doesn't need a FIFO. Protocol design: maybe typewriter text and shitty VT100 escape sequences were a bad idea? Let's use HTML-style tags, but in an endless tag soup similar to VT100 semantics, not a tree.
- AceJohnny2 6y ago
  I was thinking mostly protocol design, I suppose, with the implicit requirement that it'd have to work over various physical lines.
  - kragen 6y ago
    I have lots of ideas. But we have 55 years of experience with the advantages and disadvantages of the ADM3A, VT100, and similar terminals, and we've gradually improved it over that time. Now it's ten thousand times faster than a physical VT100, supports 24-bit color, auto-highlights URLs, has enormous scrollback, supports copy and paste, allows applications to handle mouse events, can be recorded with script or ttyrec or asciinema, can be detached and reattached later with screen or tmux or mosh, can get local echo with mosh, can share a screen and keyboard between people in different locations with screen or tmux, and has hundreds of thousands of programs already written for it — aside from backward-compatibility with teletype software like ex, ls, sh, grep, ps, ping, vmstat, and so on, you have user interfaces designed for character-cell terminals like Emacs, Vim, links, mtr, dstat, top, htop, dialog/whiptail, less, mutt, irssi, ircII, weechat, nethack, sl, robotfindskitten, command-line editing in things like tcsh and bash, progress bars in things like apt, nano, and of course the aa demo bb (if run with env -u DISPLAY). There are a lot of things that can't really be done without breaking compatibility with that ecosystem. Proportional fonts of continuously varying sizes, or padding? Now screen positions are no longer integer multiples of a character size, and you can't get things to vertically align by inserting space characters. (This is already a problem with Unicode, where different monospace fonts are monospace over different amounts of the character set; see http://canonical.org/~kragen/sw/dev3/randomlayout.py http://canonical.org/~kragen/sw/dev3/randomlayout.py for some attempts to test the limits.) Pixel graphics? Are those going to pass through tmux unharmed? (Some recent attempts to jimmy pixel graphics into terminals won't even work over ssh.) How will copy and paste work with them? Will they send another megabyte over my cellphone connection every time I press ^L to redraw the screen? We can definitely do better. But doing better is going to require a lot of experimentation to be a Pareto improvement over 55 years of tweaks to the VT100. Most of the ideas we come up with are going to turn out not to work as well in practice as we hoped they would in theory. But if you're interested in reading them anyway, some of my notes are in https://dercuano.github.io/topics/terminals.html https://dercuano.github.io/topics/terminals.html.
zamadatix 6y ago
Clean slate definitely object/packet based - just because we need to find the dimensions of the screen doesn't mean it needs to be done via control characters.
Koshkin 6y ago
And not just the terminal. Why on Earth people thought that the regular PC keyboard and the mouse required anything more complex than the good old RS-232.
wanderingjew 6y ago
Fairly easy, a few thousand lines of C. Bit of work, though. Parser is the hardest, a lot of details to get right. You don't really need a buffer for the output, because you can't type that fast. Of course, driving the display is a bit harder. I'm using a parallel tft, wrote the driver, too. Keyboard code is kludgy because I'm not using diodes for cost reduction on the bom. You can see it here: violence.works yes that's a url.
- girvo 6y ago
  I'm both amazed at what you've accomplished and a little miffed that someone beat me to it: I've been working on-and-off on a similar project for a couple of years now haha. Awesome work What display did you end up settling on, where'd you source it, and how many columns do you end up fitting? That was the biggest sticking point for me.

R0b0t1 6y ago

I've given this some thought. The way to improve the TTY is to give it channels, like SSH or RDP. This way you can transit unrelated data over the same connection. And, if you think about it, most of the complexity of a serial interface is handling mode settings and state; streams abstract that away for you. Streams get rid of the need for control characters and in band signalling. Streams could be given priority and interrupt levels.

ksherlock 6y ago
If you add a header with the channel number and data length and a CRC, it starts looking like UDP.
tyingq 6y ago
RS-485 is essentially RS-232 with channels. Never used it, but Linux supports it at some level.
- nrclark 6y ago
  This might not be accurate. RS-485 is an electrical signaling standard for carrying arbitrary binary signals. It's usually used for carrying UART data, as a longer-range / higher-speed replacement for RS-232. It has no opinion about what is transmitted, and has no concept of channels or even any particular encoding format.
  - tyingq 6y ago
    It could be the basis for a tty with multiple channels, fairly easily.
    - catblast01 6y ago
      How does that make it an RS-232 with channels? You’re confused. RS-485 just like RS-232 is an electrical specification for serial signaling (with multiple access), nothing more nor less. There are USB to 485 converters that basically operate exactly the same as their serial port counterparts and will show up as a tty device in Linux/Unix flavor or a COM port in Windows.
      - tyingq 6y ago
        It's a bus, rather than point-to-point. Schemes to address devices on the bus separately exist. You could call them "channels" even. RS-232 can't do that, as it isn't an electrical bus. Respectfully, you're confused. Since tty's are fairly well married to a serial port, adding an ioctl() (or similar) that would switch channels would seem likely. RS-485 would provide a way for that to work not just with ptys, but with real physical devices as well. With reasonable backward compatibility.
        catblast01 6y ago
        This is really reaching. Channelization as being referred to upthread in reference to SSH or RDP is a protocol level abstraction, rs485 bussing is at the electrical level for multiple physical devices and also doesn’t specify any addressing scheme, that would be specified as a higher level protocol (that could be done just as well over rs-232). Device addressing and channelization are not even logically the same thing. Ethernet as it is typically deployed is not a bus either but it is used in channelized communication (packet switched no less) just fine. Edit: responding to your edit. The point still stands this is orthogonal to the physical or electrical layer. The vast vast majority of the internet including Ethernet and fiber links are point to point and clearly they carry a tremendous number of separate streams, and a small amount of broadcast. You can run layer 2 over an RS-232C serial port, whether its Ethernet, PPP, SLIP or ATM. In Linux PPP is in fact implemented as a tty line discipline. As for RS-485, there are widely used standards such as DMX512. It is not a strongly supported idea to put this stuff in the kernel, when performance in user space is sufficient. Finally, RS-232 can do the same thing (multidrop) with electrical repeaters. See MAX3322 eg.
- kyberias 6y ago
  RS-485 in itself really isn't. It has no concept of "channels" although it can be operated as multipoint. You may be mixing this with Modbus which is typically run on RS-485.
gonzus 6y ago
A big fat YES from me. I have been looking into TUIs lately, and the state of terminal handling is ridiculously bogged down by years of cruft that, honestly, I don't give a shit about anymore. To put it succinctly: I want Alt-Home and Ctrl-Click to work on a TUI. Period. If I had superpowers, I would layer the system as a very low-level library that recognizes keyboard / mouse / tablet / microphone input hardware events and encodes them somehow (so Alt is a separate key, pressing multiple keys is handled, clicking + moving the mouse is handled, etc); then a protocol that can give you this event stream, either from a local or remote source; then a layer that maps this into higher-level events (so Alt and Home pressed together is interpreted as Alt-Home). The idea would be to have the ability to detect, even over a serial connection, any of these input events, without resorting to escape codes. I don't even know if this all makes sense; I just hate the loops you have to go through today to even try to make sense of key combinations. /rant

kragen 6y ago

Are you talking about a physical machine or a protocol design? I don't think you can abstract away the exact physical communication medium from a physical machine (or your cable won't fit in the hole, or it'll burn out your circuitry), and the protocol design probably doesn't need a FIFO. Protocol design: maybe typewriter text and shitty VT100 escape sequences were a bad idea? Let's use HTML-style tags, but in an endless tag soup similar to VT100 semantics, not a tree.

AceJohnny2 6y ago
I was thinking mostly protocol design, I suppose, with the implicit requirement that it'd have to work over various physical lines.
- kragen 6y ago
  I have lots of ideas. But we have 55 years of experience with the advantages and disadvantages of the ADM3A, VT100, and similar terminals, and we've gradually improved it over that time. Now it's ten thousand times faster than a physical VT100, supports 24-bit color, auto-highlights URLs, has enormous scrollback, supports copy and paste, allows applications to handle mouse events, can be recorded with script or ttyrec or asciinema, can be detached and reattached later with screen or tmux or mosh, can get local echo with mosh, can share a screen and keyboard between people in different locations with screen or tmux, and has hundreds of thousands of programs already written for it — aside from backward-compatibility with teletype software like ex, ls, sh, grep, ps, ping, vmstat, and so on, you have user interfaces designed for character-cell terminals like Emacs, Vim, links, mtr, dstat, top, htop, dialog/whiptail, less, mutt, irssi, ircII, weechat, nethack, sl, robotfindskitten, command-line editing in things like tcsh and bash, progress bars in things like apt, nano, and of course the aa demo bb (if run with env -u DISPLAY). There are a lot of things that can't really be done without breaking compatibility with that ecosystem. Proportional fonts of continuously varying sizes, or padding? Now screen positions are no longer integer multiples of a character size, and you can't get things to vertically align by inserting space characters. (This is already a problem with Unicode, where different monospace fonts are monospace over different amounts of the character set; see http://canonical.org/~kragen/sw/dev3/randomlayout.py http://canonical.org/~kragen/sw/dev3/randomlayout.py for some attempts to test the limits.) Pixel graphics? Are those going to pass through tmux unharmed? (Some recent attempts to jimmy pixel graphics into terminals won't even work over ssh.) How will copy and paste work with them? Will they send another megabyte over my cellphone connection every time I press ^L to redraw the screen? We can definitely do better. But doing better is going to require a lot of experimentation to be a Pareto improvement over 55 years of tweaks to the VT100. Most of the ideas we come up with are going to turn out not to work as well in practice as we hoped they would in theory. But if you're interested in reading them anyway, some of my notes are in https://dercuano.github.io/topics/terminals.html https://dercuano.github.io/topics/terminals.html.

zamadatix 6y ago

Clean slate definitely object/packet based - just because we need to find the dimensions of the screen doesn't mean it needs to be done via control characters.

Koshkin 6y ago

And not just the terminal. Why on Earth people thought that the regular PC keyboard and the mouse required anything more complex than the good old RS-232.

wanderingjew 6y ago

Fairly easy, a few thousand lines of C. Bit of work, though. Parser is the hardest, a lot of details to get right. You don't really need a buffer for the output, because you can't type that fast. Of course, driving the display is a bit harder. I'm using a parallel tft, wrote the driver, too. Keyboard code is kludgy because I'm not using diodes for cost reduction on the bom. You can see it here: violence.works yes that's a url.

girvo 6y ago
I'm both amazed at what you've accomplished and a little miffed that someone beat me to it: I've been working on-and-off on a similar project for a couple of years now haha. Awesome work What display did you end up settling on, where'd you source it, and how many columns do you end up fitting? That was the biggest sticking point for me.