6 ms·
How stdbuf works
- tptacek 4y agoThis was a fun read! It started out "TIL! Where has this `stdbuf` been all my life?", but then turned to "jfc, if that's how it works, no wonder I've never heard of this".
- deleted 4y ago[deleted]
- tkhattra 4y agodjb had a similar tool called 'nobuf', released as part of his ptyget package [0] way back on comp.sources.unix during the usenet era [0] http://jdebp.info/Softwares/djbwares/bernstein-ptyget.html http://jdebp.info/Softwares/djbwares/bernstein-ptyget.html
- arberx 4y agoGreat article! You can also use std::flush to get this behavior by default in the program.
- yjftsjthsd-h 4y agoI wonder if it could add support for pretending to be a tty for the benefit of programs where the current approach doesn't work (ex. static binaries).
- cstrahan 4y agoThat’s what unbuffer does, as mentioned in the article.
- doorman2 4y agoI was recently working with a bunch of pseudo-terminal stuff, so I was curious about how unbuffer works. One thing that stuck out to me in the manpages is this: Caveats unbuffer -p may appear to work incorrectly if a process feeding input to unbuffer exits. Consider: process1 | unbuffer -p process2 | process3 If process1 exits, process2 may not yet have finished. It is impossible for unbuffer to know long to wait for process2 and process2 may not ever finish, for example, if it is a filter. For expediency, unbuffer simply exits when it encounters an EOF from either its input or process2. Why does unbuffer care whether or not process2 exits? Can't it receive the EOF from process1, send it to process2, and then wait for process2 to exit? If process2 doesn't exit after receiving an EOF from stdin, then what's the issue about simply letting the pipeline hang? Isn't that what would happen without unbuffer?
- mzs 4y agofrom the expect FAQ, of which unbuffer is a part: #43. Why doesn't Expect kill telnet (or other programs) sometimes? From: libes (Don Libes) To: Karl.Sierka@Labyrinth.COM Subject: Re: need help running telnet Expect script from cron on sunos 4.1.3 karl.sierka@labyrinth.com writes: > The only problem I am still having with the script I wrote is that > the telnet does not seem to die on it's own, unless I turn on debugging. Actually, Expect doesn't explicitly kill processes at all. Generally, processes kill themselves after reading EOF on input. So it just seems like Expect kills all of its children. > I was forced to save the pid of the spawned telnet, and kill it with an > 'exec kill $pid' in a proc that is hopefully called before the script > exits. This seems to work fine, but it makes me nervous since omnet > charges for connect time, and leaving a hung telnet lying around could > get expensive. I warned the rest of the staff so that they will also be > on the lookout for any possible hung telnets to omnet. The problem is that telnet is not recognizing EOF. (This is quite understandable since real users can't actually generate one from the telnet user interface.) The solution is to either 1) explicitly drive telnet to kill itself (i.e., a graceful logout) followed by "expect eof" or 2) "exec kill" as you are doing. This is described further in Exploring Expect beginning on page 103. Don
- doorman2 4y agoThanks, that's is interesting background. From my pty experience, though, the answer is a bit unsatisfying. One reason is that I thought ^D generated an EOF in the terminal. For instance, if I use `wc` in the terminal, it reads from STDIN until I use ^D and then I get the output. Another reason is that it seems like a bug in the program to not respond to EOF. At the very least, unbuffer could offer an option? Anyway, I digress... The PTY interface in general feels a bit lacking. Normally, if you use a pipe, you can close to write end of the pipe to generate an EOF. With pseudo-terminals, you only have one file descriptor for writing input and reading output so you can't have the system generate an EOF without disabling your ability to read the output.
- syrrim 4y ago
- KerryJones 4y agoAnyone else misread "How Stuff Works"? No? Just me? Alright then...
- kazinator 4y agoI recently integrated stdbuf logic in an embedded product. Basically same logic: set the LD_PRELOAD environment variable, taking care to extend LD_PRELOAD if it already has a value. This was part of an effort to redirect the output of sub-programs such as external command executions and daemons into the application's logging system (fork and capture stdout). Problem is, some things things redirected to a pipe will fully buffer their output so you don't get the output in a timely way. I noticed that the target system has a libstdbuf.so (even though its rootfs image lacks the stdbuf program) and the rest was history.
- bxparks 4y agoTotal tangent: What is the magic on that page that renders: if (putenv (var) != 0) as if (putenv (var) ≠ 0) on my browser (Firefox)? When I try to copy that text, it copies as '!=' into my copypaste buffer.
- kec 4y agoThey're using a webfont with a ligature for !=.
- bxparks 4y agoSheeet, I didn't even know that was possible. Why would they use that font for displaying C code? I looked at that ≠ and spent a few minutes trying to figure out what language it was. Maybe I should uncheck the "Allow pages to choose their own fonts, instead of your selections above" option in Firefox, though it woulld probably break other websites.
- Arnavion 4y ago>Why would they use that font for displaying C code? Using ligature fonts for programming became popular a few years ago. As in some people even use them in their editors.
- bspammer 4y agoWhich is totally fine on their own machine, but using a webfont to force ligatures on anyone viewing their website is just confusing as a reader.
- Arnavion 4y agoNo, I get it. I don't use them either because I find them confusing. I'm just telling you you missed out on what was a quite popular trend from a few years ago.
- remram 4y ago
- skissane 4y agoHow I think it should work: associate with each file descriptor (or should I say file handle / struct file) a “buffering” state, initially blank, with IOCTLs to get/set it. In the stdio implementation, call the get IOCTL at startup on stdin/stdout/stderr, and also on any fdopen. If get IOCTL returns non-blank buffering settings, stdio adopts them for the FILE* pointing to that fd. Advantages: No mucking around with LD_PRELOAD and associated security-related restrictions. Even works for static binaries, so long as they are linked with a stdio which supports this feature (and running under a kernel with that IOCTL). (Actually, I was thinking you could even have an IOCTL to get/set arbitrary metadata attributes on a file handle - unlike xattrs, this would only be for each opening of the file, not the file on disk. As well as controlling stdio buffering, one can imagine other potential uses - for example, content type negotiation in pipelines.)