2 ms·

I was recently working with a bunch of pseudo-terminal stuff, so I was curious about how unbuffer works. One thing that stuck out to me in the manpages is this:

I was recently working with a bunch of pseudo-terminal stuff, so I was curious about how unbuffer works. One thing that stuck out to me in the manpages is this: Caveats unbuffer -p may appear to work incorrectly if a process feeding input to unbuffer exits. Consider: process1 | unbuffer -p process2 | process3 If process1 exits, process2 may not yet have finished. It is impossible for unbuffer to know long to wait for process2 and process2 may not ever finish, for example, if it is a filter. For expediency, unbuffer simply exits when it encounters an EOF from either its input or process2. Why does unbuffer care whether or not process2 exits? Can't it receive the EOF from process1, send it to process2, and then wait for process2 to exit? If process2 doesn't exit after receiving an EOF from stdin, then what's the issue about simply letting the pipeline hang? Isn't that what would happen without unbuffer?

mzs 4y ago
from the expect FAQ, of which unbuffer is a part: #43. Why doesn't Expect kill telnet (or other programs) sometimes? From: libes (Don Libes) To: Karl.Sierka@Labyrinth.COM Subject: Re: need help running telnet Expect script from cron on sunos 4.1.3 karl.sierka@labyrinth.com writes: > The only problem I am still having with the script I wrote is that > the telnet does not seem to die on it's own, unless I turn on debugging. Actually, Expect doesn't explicitly kill processes at all. Generally, processes kill themselves after reading EOF on input. So it just seems like Expect kills all of its children. > I was forced to save the pid of the spawned telnet, and kill it with an > 'exec kill $pid' in a proc that is hopefully called before the script > exits. This seems to work fine, but it makes me nervous since omnet > charges for connect time, and leaving a hung telnet lying around could > get expensive. I warned the rest of the staff so that they will also be > on the lookout for any possible hung telnets to omnet. The problem is that telnet is not recognizing EOF. (This is quite understandable since real users can't actually generate one from the telnet user interface.) The solution is to either 1) explicitly drive telnet to kill itself (i.e., a graceful logout) followed by "expect eof" or 2) "exec kill" as you are doing. This is described further in Exploring Expect beginning on page 103. Don
- doorman2 4y ago
  Thanks, that's is interesting background. From my pty experience, though, the answer is a bit unsatisfying. One reason is that I thought ^D generated an EOF in the terminal. For instance, if I use `wc` in the terminal, it reads from STDIN until I use ^D and then I get the output. Another reason is that it seems like a bug in the program to not respond to EOF. At the very least, unbuffer could offer an option? Anyway, I digress... The PTY interface in general feels a bit lacking. Normally, if you use a pipe, you can close to write end of the pipe to generate an EOF. With pseudo-terminals, you only have one file descriptor for writing input and reading output so you can't have the system generate an EOF without disabling your ability to read the output.
  - syrrim 4y ago
    ^D only does that in cooked input mode. In raw mode ^D send ^D, and it's up to the program to interpret that, or not. This is essential in, eg, telnet, since it wants to be able to open a session on a remote host that behaves exactly like a session on a local host, including its behaviour in response to ^D.
  - skissane 4y ago
    There should be an IOCTL to trigger an EOF condition on a PTY (even if it is in raw mode). So (once the input buffer is empty), read() calls on the client side will just return 0 instead of blocking. That requires code changes to the Linux (or whatever other OS you are using) kernel. An approach without any kernel code changes, for Linux, would be to use CUSE to implement a user-space character device. (FreeBSD has a CUSE too, although apparently its implementation is not very compatible with the Linux one). Note you can't actually create a custom tty using CUSE – it doesn't support registering its devices with the tty subsystem – but you can create a device which implements (yourself) enough of the TTY IOCTLs to trick other programs into thinking it is a TTY. (isatty() calls tcgetattr() which calls ioctl(TCGETS), so implementing that IOCTL is enough to convince most programs you are a TTY.)

mzs 4y ago

from the expect FAQ, of which unbuffer is a part: #43. Why doesn't Expect kill telnet (or other programs) sometimes? From: libes (Don Libes) To: Karl.Sierka@Labyrinth.COM Subject: Re: need help running telnet Expect script from cron on sunos 4.1.3 karl.sierka@labyrinth.com writes: > The only problem I am still having with the script I wrote is that > the telnet does not seem to die on it's own, unless I turn on debugging. Actually, Expect doesn't explicitly kill processes at all. Generally, processes kill themselves after reading EOF on input. So it just seems like Expect kills all of its children. > I was forced to save the pid of the spawned telnet, and kill it with an > 'exec kill $pid' in a proc that is hopefully called before the script > exits. This seems to work fine, but it makes me nervous since omnet > charges for connect time, and leaving a hung telnet lying around could > get expensive. I warned the rest of the staff so that they will also be > on the lookout for any possible hung telnets to omnet. The problem is that telnet is not recognizing EOF. (This is quite understandable since real users can't actually generate one from the telnet user interface.) The solution is to either 1) explicitly drive telnet to kill itself (i.e., a graceful logout) followed by "expect eof" or 2) "exec kill" as you are doing. This is described further in Exploring Expect beginning on page 103. Don

doorman2 4y ago
Thanks, that's is interesting background. From my pty experience, though, the answer is a bit unsatisfying. One reason is that I thought ^D generated an EOF in the terminal. For instance, if I use `wc` in the terminal, it reads from STDIN until I use ^D and then I get the output. Another reason is that it seems like a bug in the program to not respond to EOF. At the very least, unbuffer could offer an option? Anyway, I digress... The PTY interface in general feels a bit lacking. Normally, if you use a pipe, you can close to write end of the pipe to generate an EOF. With pseudo-terminals, you only have one file descriptor for writing input and reading output so you can't have the system generate an EOF without disabling your ability to read the output.
- syrrim 4y ago
  ^D only does that in cooked input mode. In raw mode ^D send ^D, and it's up to the program to interpret that, or not. This is essential in, eg, telnet, since it wants to be able to open a session on a remote host that behaves exactly like a session on a local host, including its behaviour in response to ^D.
- skissane 4y ago
  There should be an IOCTL to trigger an EOF condition on a PTY (even if it is in raw mode). So (once the input buffer is empty), read() calls on the client side will just return 0 instead of blocking. That requires code changes to the Linux (or whatever other OS you are using) kernel. An approach without any kernel code changes, for Linux, would be to use CUSE to implement a user-space character device. (FreeBSD has a CUSE too, although apparently its implementation is not very compatible with the Linux one). Note you can't actually create a custom tty using CUSE – it doesn't support registering its devices with the tty subsystem – but you can create a device which implements (yourself) enough of the TTY IOCTLs to trick other programs into thinking it is a TTY. (isatty() calls tcgetattr() which calls ioctl(TCGETS), so implementing that IOCTL is enough to convince most programs you are a TTY.)