3 ms·

> programs should print “regular” output to stdout and errors to stderr This is really important. I'd like to expand on this. Standard output is for the data

> programs should print “regular” output to stdout and errors to stderr This is really important. I'd like to expand on this. Standard output is for the data the program was asked to produce, no more and no less. If user asked for some JSON data, standard output should contain that exact JSON object and absolutely nothing else. Standard "error" is actually a misnomer. It should have been called the standard user stream. Anything meant for the user to read on the terminal is supposed to go there. Error messages are of course included in that set but so are status messages and verbose output. This ensures the output of programs can be piped into other programs seamlessly. Non-output data still gets sent to the terminal or redirected somewhere else. Would have been great if programs were able to easily create new terminal-connected file descriptors for specific purposes. They could document those numbers in their manuals just like they document exit codes. Then users would get "ports" for every output. Could cut down on parsing significantly. For compatibility, they could all redirect to either standard output or standard error by default... I think I'm gonna experiment with this a bit.

packetlost 2y ago
I'm not sure I like the "standard user stream" name, but I otherewise agree with everything here.
- matheusmoreira 2y ago
  Yeah, naming is hard. If there is a better word for "non-output", I don't know it.
  - packetlost 2y ago
    I would maybe call it stdlog or stdinfo vs stderr, but yeah naming things is hard.
    - matheusmoreira 2y ago
      I think stdinfo would exclude error messages from the definition, just like stderr excludes info messages. I really like stdlog. Standard log stream is a pretty awesome name. Short and terse, the word "log" doesn't even need abbreviation and it's correct since it's a superset of error and info streams and also generic enough to cover other unforeseen categories. I'll use it from now on!
      - packetlost 2y ago
        The idea for info is that it's informational, which is always the case regardless of whether it's printing error messages due to a failure or informational messages because the user specified a `-v` flag or whatever. Whether the command failed or not is communicated via the exit code. That being said, I do prefer stdlog for the reasons: it communicates the intent and usage as unambiguously and tersely as possible.
        matheusmoreira 2y ago
        > because the user specified a `-v` flag or whatever The way I see it is the version information and help text belong on standard output if the user passes --version and --help since that's what the user explicitly asked for. When the command is invoked incorrectly, the help text should go to standard error while standard output should be empty. I agree about the exit code. People like to parse error messages and that's always wrong. Everything should be done via exit codes instead.
        packetlost 2y ago
        The `-v` was in reference to the common convention of one or more `v` flags increasing the verbosity of logging.
    - ryandrake 2y ago
      C++ has std::clog[1], but it's basically cerr, and the OS still only knows about stdin, stdout, and stderr. 1: https://cplusplus.com/reference/iostream/clog/ https://cplusplus.com/reference/iostream/clog/
- paulddraper 2y ago
  I'd name it "standard log." That's what it is....a log. Of errors, warnings, informational messages, whatever to complement the primary output. BONUS: stdlog fits nicely :)
- saghm 2y ago
  What about "debug"? I think the important part about stderr is that it's "meta" info about how the program is running (or ran) rather than what the program is intended to produce as output. This seems pretty similar to what often is referred to as the "debug" level of logging (which includes lower levels of logging like warnings and errors).
lionkor 2y ago
Unless your app doesn't generate output,then grepping the output will not work if it's stderr and it will be confusing
- bombcar 2y ago
  You can learn the 2>&1 thing, but it would be nice if there was a feature of one of the greps to "slurp up" the error output. Of course, the real problem is there's no standard, the standards that do exist are ignored, and each new command-line tool generates a new standard.
  - sudobash1 2y ago
    It generally isn't possible for grep to "slurp up" the error output because stderr does not get passed through pipes (by default). This is shell behavior and grep cannot do anything about it. As a side note, some shells have started implementing a shorthand for `foo 2>&1 | grep` which is `foo |& grep`.
    - ElevenLathe 2y ago
      Probably you could do a hack like finding the program at the other end of the pipe and looking at its open file descriptors, but lots of corner cases to consider.
    - kelnos 2y ago
      Right, the problem is that apparently shells haven't seen user need for selectively redirecting one or the other. Like I might want to see foo's stdout on the terminal, but pipe stderr (and only stderr) to grep, perhaps so I only see some specific error messages that I care about. `2>&1` will send stderr to stdout, and then the pipe will send everything to grep (and I believe `|&` does the same). (To be fair, that does seem like a pretty rare use case.)
      - sudobash1 2y ago
        Actually, you can. If you run: foo 2>&1 >&- | grep ... then you will grep only the stderr. The stdout will be discarded. Alternatively, if you only want to grep to see the stderr, but you still want to see stdout, you can swap stderr and stdout like this: foo 3>&2 2>&1 1>&3 | grep ... And in many shells, you can easily split the stdout and stderr into two separate pipelines like this: foo > >(grep stdout) 2> >(grep stderr)
AndrewDucker 2y ago
This is definitely one of the things that PowerShell got right. 6 different streams, each of which can be intercepted separately and configured differently. https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_output_streams?view=powershell-7.4 https://learn.microsoft.com/en-us/powershell/module/microsof...
- matheusmoreira 2y ago
  There's a lot of good ideas in power shell. I like how they broke up the log stream into separate streams by severity. It still seems arbitrary and insufficiently general though. Why severity and not some other criteria? What if programs could define any number of output streams? By default they could all coalesce into the terminal but other programs could connect to each one separately if they needed. Like an audio mixer of sorts: by default you get the mixed audio but there are ways to access each individual voice if needed.
- immibis 2y ago
  I don't think that 6 different streams that will always go to the same place but some of them are usually turned off are very useful. Standard-error/user is useful because it goes to a different place. Levels would be better suited to a simple global variable, or a level associated with each printed line. The chance that someone redirects errors to error.log is reasonable, but the chance that someone redirects errors to error.log and warnings to warning.log and debug to debug.log and verbose to verbose.log is pretty low.
  - AndrewDucker 2y ago
    I turn on Verbose logging sometimes, to get more info. Error logging can be set to either spit out the error and continue, or to stop processing, which is very useful. Warning and error messages come out in their own colours automatically, which is nice. And sure, you can just have them all come out to the same place. But having the options to configure them is very handy.
    - 2y ago

packetlost 2y ago

I'm not sure I like the "standard user stream" name, but I otherewise agree with everything here.

matheusmoreira 2y ago
Yeah, naming is hard. If there is a better word for "non-output", I don't know it.
- packetlost 2y ago
  I would maybe call it stdlog or stdinfo vs stderr, but yeah naming things is hard.
  - matheusmoreira 2y ago
    I think stdinfo would exclude error messages from the definition, just like stderr excludes info messages. I really like stdlog. Standard log stream is a pretty awesome name. Short and terse, the word "log" doesn't even need abbreviation and it's correct since it's a superset of error and info streams and also generic enough to cover other unforeseen categories. I'll use it from now on!
    - packetlost 2y ago
      The idea for info is that it's informational, which is always the case regardless of whether it's printing error messages due to a failure or informational messages because the user specified a `-v` flag or whatever. Whether the command failed or not is communicated via the exit code. That being said, I do prefer stdlog for the reasons: it communicates the intent and usage as unambiguously and tersely as possible.
      - matheusmoreira 2y ago
        > because the user specified a `-v` flag or whatever The way I see it is the version information and help text belong on standard output if the user passes --version and --help since that's what the user explicitly asked for. When the command is invoked incorrectly, the help text should go to standard error while standard output should be empty. I agree about the exit code. People like to parse error messages and that's always wrong. Everything should be done via exit codes instead.
        packetlost 2y ago
        The `-v` was in reference to the common convention of one or more `v` flags increasing the verbosity of logging.
  - ryandrake 2y ago
    C++ has std::clog[1], but it's basically cerr, and the OS still only knows about stdin, stdout, and stderr. 1: https://cplusplus.com/reference/iostream/clog/ https://cplusplus.com/reference/iostream/clog/
paulddraper 2y ago
I'd name it "standard log." That's what it is....a log. Of errors, warnings, informational messages, whatever to complement the primary output. BONUS: stdlog fits nicely :)
saghm 2y ago
What about "debug"? I think the important part about stderr is that it's "meta" info about how the program is running (or ran) rather than what the program is intended to produce as output. This seems pretty similar to what often is referred to as the "debug" level of logging (which includes lower levels of logging like warnings and errors).

lionkor 2y ago

Unless your app doesn't generate output,then grepping the output will not work if it's stderr and it will be confusing

bombcar 2y ago
You can learn the 2>&1 thing, but it would be nice if there was a feature of one of the greps to "slurp up" the error output. Of course, the real problem is there's no standard, the standards that do exist are ignored, and each new command-line tool generates a new standard.
- sudobash1 2y ago
  It generally isn't possible for grep to "slurp up" the error output because stderr does not get passed through pipes (by default). This is shell behavior and grep cannot do anything about it. As a side note, some shells have started implementing a shorthand for `foo 2>&1 | grep` which is `foo |& grep`.
  - ElevenLathe 2y ago
    Probably you could do a hack like finding the program at the other end of the pipe and looking at its open file descriptors, but lots of corner cases to consider.
  - kelnos 2y ago
    Right, the problem is that apparently shells haven't seen user need for selectively redirecting one or the other. Like I might want to see foo's stdout on the terminal, but pipe stderr (and only stderr) to grep, perhaps so I only see some specific error messages that I care about. `2>&1` will send stderr to stdout, and then the pipe will send everything to grep (and I believe `|&` does the same). (To be fair, that does seem like a pretty rare use case.)
    - sudobash1 2y ago
      Actually, you can. If you run: foo 2>&1 >&- | grep ... then you will grep only the stderr. The stdout will be discarded. Alternatively, if you only want to grep to see the stderr, but you still want to see stdout, you can swap stderr and stdout like this: foo 3>&2 2>&1 1>&3 | grep ... And in many shells, you can easily split the stdout and stderr into two separate pipelines like this: foo > >(grep stdout) 2> >(grep stderr)

AndrewDucker 2y ago

This is definitely one of the things that PowerShell got right. 6 different streams, each of which can be intercepted separately and configured differently. https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_output_streams?view=powershell-7.4 https://learn.microsoft.com/en-us/powershell/module/microsof...

matheusmoreira 2y ago
There's a lot of good ideas in power shell. I like how they broke up the log stream into separate streams by severity. It still seems arbitrary and insufficiently general though. Why severity and not some other criteria? What if programs could define any number of output streams? By default they could all coalesce into the terminal but other programs could connect to each one separately if they needed. Like an audio mixer of sorts: by default you get the mixed audio but there are ways to access each individual voice if needed.
immibis 2y ago
I don't think that 6 different streams that will always go to the same place but some of them are usually turned off are very useful. Standard-error/user is useful because it goes to a different place. Levels would be better suited to a simple global variable, or a level associated with each printed line. The chance that someone redirects errors to error.log is reasonable, but the chance that someone redirects errors to error.log and warnings to warning.log and debug to debug.log and verbose to verbose.log is pretty low.
- AndrewDucker 2y ago
  I turn on Verbose logging sometimes, to get more info. Error logging can be set to either spit out the error and continue, or to stop processing, which is very useful. Warning and error messages come out in their own colours automatically, which is nice. And sure, you can just have them all come out to the same place. But having the options to configure them is very handy.
  - 2y ago