10 ms·

IMO, this is an anti-pattern. It's violates the principle of least surprise. (How come I see X when I run the command, but I can't grep for X in its output?

• burke 12y ago
  I think it depends what sort of things you use it for. I often use it to switch on or off ANSI colourization, which doesn't really violate the principle of least surprise. When used sparingly and thoughtfully, I've never personally had an issue with it.
  • dap 12y ago
    Good point, and it makes me even more convinced that all rules of thumb have important exceptions. In fact, I've also used tools that use ANSI colorization, and disable that when not talking to a tty, and wished that they wouldn't because I was piping them to "less -R" :) (At least that's a graceful failure mode, and it's pretty clear what probably happened even if one doesn't exactly understand the details.)
    • burke 12y ago
      I've always thought it would be nice to have a utility like cat that I could pipe commands to, which would trick them into thinking all their streams were attached to a tty, so you could do "uses_colours | ttycat | less -R". I'm sure it's possible, but you'd have to acquire a new pty and decide what termios settings you want. It's a nontrivial hack, I think. I'm actually kind of surprised it's not in moreutils[1]. [1]: https://joeyh.name/code/moreutils/ https://joeyh.name/code/moreutils/ EDIT: Hmm, maybe it's not possible. I can't figure out exactly how to do it, anyway. EDIT again: Apparently `script` does this on Linux: http://monosnap.com/image/Qlig4CHmQgV9pxvSmndVUgMTU88Adz http://monosnap.com/image/Qlig4CHmQgV9pxvSmndVUgMTU88Adz EDIT again: Expect's `unbuffer` also does it: http://expect.sourceforge.net/example/unbuffer.man.html#toc http://expect.sourceforge.net/example/unbuffer.man.html#toc
      • dnlrn 12y ago
        It’s not possible in the way you think, because the two programs don’t know about each other. All the pipe character does is put the stdout into the stdin of the other program. So this feature must actually be present in the shell and maybe it is. I’m no expert but maybe zsh already offers something like this?
      • JdeBP 12y ago
        > I've always thought it would be nice to have a utility like cat that I could pipe commands to, which would trick them into thinking all their streams were attached to a tty, Daniel J. Bernstein wrote a "pty" package back around 1991 that did this. Version 4 of the package was published in 1992 to comp.sources.unix (volume 25 issues 127 to 135). It's still locatable on the World Wide Web. Bernstein later updated this, around 1999, with a "ptyget" package that was more modular and that had the session management commands moved out of the toolset to elsewhere. The command from that package to do exactly what you describe is "ptybandage". There is also "ptyrun". Paul Jarc still publishes a fixed version of ptyget (that attempts to deal with the operating-system-specific pseudo-terminal device ioctls in the original) at http://code.dogmap.org./ptyget/ http://code.dogmap.org./ptyget/ . As a bonus feature for people who use source code, there are similar "ptybandage" and "ptyrun" scripts, for which you will need Laurent Bercot's execline tool (http://skarnet.org./software/execline/ http://skarnet.org./software/execline/), in the source archive for the nosh package at http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh.html http://homepage.ntlworld.com./jonathan.deboynepollard/Softwa... . These make use of the terminal-management tools in the nosh toolset. With both of these, you should be able to run "ptybandage uses_colours_if_tty | less -R"
  • chriswarbo 12y ago
    You may not have issue with the sort of things you use it for, but others might. For example, I run shells in Emacs and have had to tweak loads of shell scripts written by colleagues to fix their poorly-implemented colourisation. It's useful to know when a test has failed; it's not so useful to have the whole terminal set to white text on a pale pink background. One day I couldn't SSH into our servers from Emacs. It turned out somebody had edited .bashrc for the admin user to make the bash prompt blue. Emacs' TRAMP process was looking for a prompt ending in "$" or "#", not "$\[\033[0m\]", so it didn't realise the connections were successful. There are two ways of handling this: we can blame the source of the bug (the person adding the colours incorrectly, or the assumption-loaded TRAMP regex), but there will always be more bugs in situations we'd never think of. Alternatively, we can avoid being 'too clever', and instead aim for consistency and least surprise.
    • philh 12y ago
      Are you suggesting that colored prompts violate the rules of consistency and least surprise? (Actually, if you are suggesting that, I'm not going to disagree. But I am going to say that if so, those rules don't apply in the case of colored prompts, because colored prompts are useful.)
      • chriswarbo 12y ago
        I suppose I'm suggesting that, aside from personal scripts, we shouldn't assume too much about who our users are and what they're trying to do. The principle of least power tells us to use the dumbest format that will work, eg. plain text. Anything we add on top of that, eg. ANSI colour codes, will be useful to some but harmful to others. The tricky part is working out which of those categories the current user is in.
        • philh 12y ago
          So is your proposed solution not to have colored prompts? (I vehemently disagree.) Or not to put them in a .bashrc? (I still disagree, but only strongly.) Or something else?
• gohrt 12y ago
  This behavior of Unix programs is basically the same concept as Perl's "context" (list vs scalar), but even moreso.
• userbinator 12y ago
  I agree, especially for the behaviour from the parent: "ps -f" truncates long lines instead of wrapping, while "ps -f | cat" lets the long lines live How people usually discover what these commands do is by running them interactively, and if that results in some output being hidden vs being run noninteractively, then they have little reason to believe that it could yield more output than what they're used to seeing. I think a certain number of "ps" users don't know it can display full paths and commands, if they've only ever used it interactively.
  • edwintorok 12y ago
    On a quick look through ps's manpage I couldn't find anything about this. Am I missing something?
    • mzs 12y ago
      -w
      • edwintorok 12y ago
        It doesn't say that it'll do that automatically when run through a pipe.
        • mzs 12y ago
          Yep, and the result is I have needless sprinklings of "www" in my shell scripts cause of habit. Technically I don't need it the moment I pipe into grep, but oh well ;) Anyway I personally dislike the SysV-like 'I need to add stuff like -aef' to get all that on-the-screen/into-grep vs the BSD-like it knows about TTY but I can convince it otherwise into stuf like 'less -S' - personal taste I guess.
          • edwintorok 12y ago
            Actually I just found this in the 'ps' manual, it looks like the output width is actually undefined! "If ps can not determine display width, as when output is redirected (piped) into a file or another command, the output width is undefined (it may be 80, unlimited, determined by the TERM variable, and so on)."
• vog 12y ago
  Ideed. I've seen people over and over stumbling over this weird behaviour. It may have some merits, but as a general advice this is definitely an anti-pattern. Another example is "curl", where "curl URL >outfile" is chatty on stderr, while "curl URL" is quiet on stderr. That's very annoying for scripting, you easily forget to set "-s" in your scripts due to that behaviour.
• teddyh 12y ago
  Indeed, the GNU Coding Standards explicitly argues against doing that: “please don’t make the behavior of a command-line program depend on the type of output device it gets as standard output or standard input.”¹ ① https://www.gnu.org/prep/standards/standards.html#User-Interfaces https://www.gnu.org/prep/standards/standards.html#User-Inter...
  • tieTYT 12y ago
    Interesting. Doesn't git (a pretty new tool) violate this? I think running log from the terminal pauses per page of output but if you pipe it to something it pipes all the content.
    • dkuntz2 12y ago
      I think it's just piping it to another program. It's still giving you the same output, but sending you to a program meant for humans to be able to read text in a console, instead of just printing it all out. That said, I'm not entirely sure which git pipes to.
• oakwhiz 12y ago
  I have run into this problem when trying to automate certain tasks on UNIX boxes. Dealing with programs that act differently depending on their output device is very annoying.
• easytiger 12y ago
  Yea most commands don't do this though. Most commonly used for colouring output. ls is a bit more than just a command though. It's part of the furniture and prehistoric.
• emmelaich 12y ago
  And yet .. programs are not just for composition. They have to behave sensibly for people. I love that 'git log' outputs in a pager. 'svn log' by comparison is nuts.
  • aidos 12y ago
    That's an interesting case. Should it not just output to stdout and you have to pipe it to less? If you wanted it to always pipe to less you could just set up an alias for it.
    • kec 12y ago
      The git diff and log commands are primarily intended for human consumption. I'm not the one you were relying to, but I think it makes sense that git defaults to make that consumption easier, even if it isn't strictly "Unix". (You can also disable this behavior and get a more Unix-y interface by default via gitconfig or on a case by case basis with --no-pager).
      • aidos 12y ago
        For sure, and in fact, I very happy with the current behaviour. I guess the pager is a case where it doesn't matter much if it's included or not since I guess it doesn't get in the way if you pipe to something else. Does it consume extra resources? git log | wc vs git log --no-pager | wc I'm sure it's neither here nor there in practice. More of a hypothetical question.
    • pjc50 12y ago
      I've just realised that the best solution to this (which I've never seen) is for the shell or terminal emulator to capture long output into a pager for you once it exceeds a certain length.
      • dkuntz2 12y ago
        I think there may be some flags in pagers to do this for you, but I can't remember for sure.
• pessimizer 12y ago
  I despise it when commands do this - mysql -e results are formatted differently depending on whether the output is directed to the terminal or to a file.