5 ms·
How to write to stderr so people will like you
- gte910h 16y agoThis is a nice paradigm when it works, however for programs that are often piped to other programs, I believe this can run into issues with the flush on stdout blocking. Am I wrong here?
- Periodic 16y agoWon't it just get buffered into the other program's input buffer? I believe pipes are block-buffered and basically look just like redirecting to a file.
- jrockway 16y agoIf the process you're piping to never reads stdin and your process writes to stdout with a blocking write, you will eventually be blocked. But just use an event loop for stdout, and all will be well. (That way your program is in control of its own buffering and blocking. select is probably good enough in this case.) Anyway, if you want to see freakin' echo block because of this, try: $ mkfifo foo $ echo "OH HAI" > foo <blocks> In another xterm: $ cat foo <echo unblocks and exits>.
- btilly 16y agoThe potential problem comes when stdout and stderr are going to different places. For instance stdout is going to the next stage in a pipeline while stderr is going to the console. Then the error message winds up waiting for the buffering program.
- gte910h 16y agoYes exactly. This is the scenario I'm worried about. It's pretty common actually (think logging, etc)
- scott_s 16y agoI looked up the man page for C's fflush(). One of the error messages is EPIPE, "An attempt is made to write to a pipe or FIFO that is not open for reading by any process. A SIGPIPE signal shall also be sent to the thread." So, at least in C, I think flushing is intelligent enough not to block if no one's receiving the output.
- gte910h 16y agoI'm not talking about no one receiving output. I'm talking about this case: You write a program called "thefirstprogram". Who cares what it does. However, a user is pushing its (standard) output into a second program called "asecondprogram". Now imagine "asecondprogram" has melted, but not terminated. It's basically in a screwed up state. So the run command was: bash-2.04> thefirstprogram | asecondprogram I believe the flushing scheme advocated in the article is to be considered harmful, as you will in fact never see an error message if asecondprogram gets in a bad state and stops reading from stdin (and the buffers fill up). I do agree the flushing makes for nicer ordering of error messages, but is more fragile. Only used flushed output when you really need flushed output otherwise you'll get less resilience in the case of meltdown of cooperating programs.
- ars 16y agoWhat?! If the second program is messed up, it makes no difference if you flush or not. You'll still never see an error message. Stdin will fill up, flush or not, the program will block, and then sit there - no error message. You could maybe detect if stdin is blocked and say something on stderr - but if you want to do that it makes no difference whatsoever if you flush stdout or not.
- gte910h 16y agoNo, stderr isn't being piped with that command. So you will see it printed out on the command line. You have to explicitly pipe stderr as well if you don't want this behavior
- 16y ago
- ars 16y agoIf that happens, it's supposed to block, so let it. So yes, you are wrong here.
- zokier 16y agoimho better way to solve this 'problem' would be to write error messages in such way that you don't need stdout as context to interpret them.
- crc5002 16y agoIf you can't recompile the program, an expect script will come very handy: $ ./unbuffer ./tst >/tmp/logit 2>&1; cat /tmp/logit line 1 stderr line 1 line 2 stderr line 2 http://expect.nist.gov/example/unbuffer http://expect.nist.gov/example/unbuffer
- j_baker 16y ago"(Note that programs that run other programs need to do more than just this; they need to flush stdout and perhaps stderr before they start another program.)" Could someone explain exactly why this is?
- mbreese 16y agoBecause you want to make sure your output is sent before the possible output from a child program. This is the same principle as from the article.
- lil_cain 16y agoBecause if they start another program without flushing their buffers, you'll have the output from the new program before the output from the old program, despite the fact that chronologically, the errors were generated before it started.
- CamperBob 16y agoI'm in the habit of using doing a setbuf(stdout,NULL) in the beginning of all my console apps, anyway. Buffered console I/O is something that made sense 20 years ago but certainly not anymore. If you did a setbuf(stderr,NULL) as well as setbuf(stdout,NULL), you'd achieve the same effect that the author's going for, with no need to remember to flush.
- nitrogen 16y agoIt really depends on how much data you need to write to stdout. If you're writing a single character at a time with no buffering, many terminal programs (GNOME terminal, etc.) will struggle to put out more than a few dozen lines per second, which can significantly slow down your program (I've known people who redirect their cc's stdout to /dev/null to avoid terminal latency slowing the build).
- ready 16y ago"flush stdout, write your message to stderr, then flush stderr." Standard Error is unbuffered: http://www.opengroup.org/onlinepubs/009695399/functions/stdin.html http://www.opengroup.org/onlinepubs/009695399/functions/stdi... "When opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device."
- JoachimSchipper 16y agoErm... if the problem is that stdout may be fully buffered, just set it to line buffered explicitly? (See setbuf(3) and friends.) (Note that only stdout is the issue, see ready's comment.)
- kqueue 16y agoa good practice is not to combine stdout and stderr into the same output when the ordering matters. You'll get a different result depending on the shell you are using.