4 ms·

Theres no way you can build a terminal emulator without a PTY and still expect terminal UIs (eg stuff that uses ncursors) to work. Moreover, the session state

Theres no way you can build a terminal emulator without a PTY and still expect terminal UIs (eg stuff that uses ncursors) to work. Moreover, the session state saving is not possible either since you need to save the internal state of running programs as well as their cached output (saving the output is really very easy but it's the least of your worries given the specifications you've described). You might also find this problem is better solved with a $SHELL replacement rather than a terminal emulator.

burfog 8y ago
Yes you can. I want to hook this up to an emulator (like qemu or MAME) with an emulated serial port. That is the PTY equivalent. I'm writing this emulator. So the emulator gets bytes from the emulated virtual machine, and then what? This needs to get fed into a terminal. While it is possible to invoke xterm with the very buggy -S option, there is no way for the emulator to suck the state back out of xterm for coherent snapshots. Simics uses xterm with the -S option. It sucks. A library interface is required. The whole point of a library interface is to allow session state saving. Using a $SHELL replacement is way off. That has nothing to do with the problem. So, if you were hacking on an emulator (qemu, MAME, Simics, VirtualBox, etc.) and you were trying to emulate a device with a serial port or even a modem, how would you get snapshots to contain the terminal state? To make this work, you need that state within the emulator. Passing stuff over a PTY is not going to work.
- Liquid_Fire 8y ago
  Just capture the output from the serial port in a buffer inside the emulator, and use that to save snapshots. You can then also forward any output to a file/terminal emulator/whatever is convenient, but relying on the terminal emulator to save part of your emulator's state seems weird to me, unless you are going to ship the two bundled together.
  - burfog 8y ago
    Terminal state is far more than just text on the screen. For example, an escape code could change the color. After that, there could be megabytes of output created over many hours. Loading a snapshot should not require replaying all that data. The point of a library is that the two are more than just bundled together. They are linked together, making one executable. The terminal window is a part of the emulator.
    - laumars 8y ago
      Your arguements don't really make any sense with regards to the technology you're discusing. Eg why would we replay past TTY output? Also escape are inlined so they would also be captured via the parents method. In fact contrary to your point, TTYs effectively are just text on a screen. The only thing that wouldn't be inlined via control characters nor escape sequences would be the PTY operating mode (which are defined via kernal syscalls). Im talking about stuff like local echo, flow control, how CR/LF characters are handled, etc. However that's solved with a multiplexer (screen or tmux type thing) or a custom shell, thus you could save that running state and reset it when needed (shells actually already need to do this to some extent so they can pass control of the TTY to forked processes. But we'd obviously need to take things a step further and store any state changes made by forked processes as well). However now we are back to my original solution you dismissed.
      - burfog 8y ago
        I'll go through what happens without replaying. The emulator runs, and the guest OS inside it boots up. For this example, suppose it is DOS running a multi-user BBS with 2 modems. This is not a UNIX clone; there is no /dev and there is no shell. The emulator needs to pop up 2 terminal windows so that we can log in to the BBS. We do that, and a colorful menu is displayed on each of them. At this point, we save a snapshot and shut down the host computer. Later, we start the host computer, then start the emulator, and then try to resume the snapshot. Each terminal window remains blank, which is not the correct state. The guest OS (a DOS BBS) is in the state where it is showing a colorful menu, but we can't see it at all. Our terminals, being separate from the emulator and without an ability to save/restore snapshots, are inconsistent with the emulated guest OS. In the above example, the guest OS (a DOS BBS) never did any kernel syscalls related to PTY operating mode. It directly acted on serial port hardware. There is no PTY, and adding one (where?) doesn't help. Fundamentally, the emulator needs to be able to save/restore all terminal state. That even includes stuff like incomplete escape sequences, such as when just the first byte of the escape sequence has been passed to the terminal.
- laumars 8y ago
  You're only solving half of the problem (and doing it in a needlessly complicated way at that). As I said, capturing the TTY output is the easy part. Storing the running state of the program that's writing to the TTY is your bigger issue. That's why I suggested a custom $SHELL (I guess there's no need to reinvent the command line so you could probably use a multiplexer instead?) that could manage the running state of the executing programs and their PTY.
  - burfog 8y ago
    Storing the running state of the program that's writing to the... serial port or whatever... is a solved issue. Most emulators and virtual machines can do this. For example, VMWare Workstation can do it. Simics can do it. Qemu can do it. The state of an entire OS, with all running programs, gets saved. It's more complete than when a laptop/notebook does suspend-to-disk. The emulator saves the CPU registers, the RAM, the disks, the parallel port mode, etc. The trouble is that a connected terminal is not included in that list.
    - laumars 8y ago
      There isn't anything writing to the serial port. The process of writing to the serial port is fast enough that the IO isnt the problem. What you're after is the capture of stuff that has already been outputted on the terminal and that can be captured. It's called a scroll back buffer history. Some terminals can even make that persistent. You keep trying to reinvent terminals with features that already exist to solve a problem that needs to be fixed in the server itself.
      - burfog 8y ago
        This whole conversation is weird, and I'm really not sure what is missing from my explanation. In case you missed it, I am an author of an emulator. I write the emulation, and my emulator supports guest software that needs terminals. The serial port is an emulated 16550 chip. It is much more than a PTY. It has clock divider registers, an interrupt cause register, a transmit holding register, etc. Scroll back buffer history won't let me untype something. If I take a snapshot, then a character is output to the terminal, and then I load that snapshot, the character needs to disappear from the screen. Of course, "character" could be something more destructive, like a scrolling operation or the clearing of a line.

burfog 8y ago

Yes you can. I want to hook this up to an emulator (like qemu or MAME) with an emulated serial port. That is the PTY equivalent. I'm writing this emulator. So the emulator gets bytes from the emulated virtual machine, and then what? This needs to get fed into a terminal. While it is possible to invoke xterm with the very buggy -S option, there is no way for the emulator to suck the state back out of xterm for coherent snapshots. Simics uses xterm with the -S option. It sucks. A library interface is required. The whole point of a library interface is to allow session state saving. Using a $SHELL replacement is way off. That has nothing to do with the problem. So, if you were hacking on an emulator (qemu, MAME, Simics, VirtualBox, etc.) and you were trying to emulate a device with a serial port or even a modem, how would you get snapshots to contain the terminal state? To make this work, you need that state within the emulator. Passing stuff over a PTY is not going to work.

Liquid_Fire 8y ago
Just capture the output from the serial port in a buffer inside the emulator, and use that to save snapshots. You can then also forward any output to a file/terminal emulator/whatever is convenient, but relying on the terminal emulator to save part of your emulator's state seems weird to me, unless you are going to ship the two bundled together.
- burfog 8y ago
  Terminal state is far more than just text on the screen. For example, an escape code could change the color. After that, there could be megabytes of output created over many hours. Loading a snapshot should not require replaying all that data. The point of a library is that the two are more than just bundled together. They are linked together, making one executable. The terminal window is a part of the emulator.
  - laumars 8y ago
    Your arguements don't really make any sense with regards to the technology you're discusing. Eg why would we replay past TTY output? Also escape are inlined so they would also be captured via the parents method. In fact contrary to your point, TTYs effectively are just text on a screen. The only thing that wouldn't be inlined via control characters nor escape sequences would be the PTY operating mode (which are defined via kernal syscalls). Im talking about stuff like local echo, flow control, how CR/LF characters are handled, etc. However that's solved with a multiplexer (screen or tmux type thing) or a custom shell, thus you could save that running state and reset it when needed (shells actually already need to do this to some extent so they can pass control of the TTY to forked processes. But we'd obviously need to take things a step further and store any state changes made by forked processes as well). However now we are back to my original solution you dismissed.
    - burfog 8y ago
      I'll go through what happens without replaying. The emulator runs, and the guest OS inside it boots up. For this example, suppose it is DOS running a multi-user BBS with 2 modems. This is not a UNIX clone; there is no /dev and there is no shell. The emulator needs to pop up 2 terminal windows so that we can log in to the BBS. We do that, and a colorful menu is displayed on each of them. At this point, we save a snapshot and shut down the host computer. Later, we start the host computer, then start the emulator, and then try to resume the snapshot. Each terminal window remains blank, which is not the correct state. The guest OS (a DOS BBS) is in the state where it is showing a colorful menu, but we can't see it at all. Our terminals, being separate from the emulator and without an ability to save/restore snapshots, are inconsistent with the emulated guest OS. In the above example, the guest OS (a DOS BBS) never did any kernel syscalls related to PTY operating mode. It directly acted on serial port hardware. There is no PTY, and adding one (where?) doesn't help. Fundamentally, the emulator needs to be able to save/restore all terminal state. That even includes stuff like incomplete escape sequences, such as when just the first byte of the escape sequence has been passed to the terminal.
laumars 8y ago
You're only solving half of the problem (and doing it in a needlessly complicated way at that). As I said, capturing the TTY output is the easy part. Storing the running state of the program that's writing to the TTY is your bigger issue. That's why I suggested a custom $SHELL (I guess there's no need to reinvent the command line so you could probably use a multiplexer instead?) that could manage the running state of the executing programs and their PTY.
- burfog 8y ago
  Storing the running state of the program that's writing to the... serial port or whatever... is a solved issue. Most emulators and virtual machines can do this. For example, VMWare Workstation can do it. Simics can do it. Qemu can do it. The state of an entire OS, with all running programs, gets saved. It's more complete than when a laptop/notebook does suspend-to-disk. The emulator saves the CPU registers, the RAM, the disks, the parallel port mode, etc. The trouble is that a connected terminal is not included in that list.
  - laumars 8y ago
    There isn't anything writing to the serial port. The process of writing to the serial port is fast enough that the IO isnt the problem. What you're after is the capture of stuff that has already been outputted on the terminal and that can be captured. It's called a scroll back buffer history. Some terminals can even make that persistent. You keep trying to reinvent terminals with features that already exist to solve a problem that needs to be fixed in the server itself.
    - burfog 8y ago
      This whole conversation is weird, and I'm really not sure what is missing from my explanation. In case you missed it, I am an author of an emulator. I write the emulation, and my emulator supports guest software that needs terminals. The serial port is an emulated 16550 chip. It is much more than a PTY. It has clock divider registers, an interrupt cause register, a transmit holding register, etc. Scroll back buffer history won't let me untype something. If I take a snapshot, then a character is output to the terminal, and then I load that snapshot, the character needs to disappear from the screen. Of course, "character" could be something more destructive, like a scrolling operation or the clearing of a line.