7 ms·

I find it easier to understand in terms of the Unix syscall API. `2>&1` literally translates as `dup2(1, 2)`, and indeed that's exactly how it works. In the cla

by wahern 8mo ago

I find it easier to understand in terms of the Unix syscall API. `2>&1` literally translates as `dup2(1, 2)`, and indeed that's exactly how it works. In the classic unix shells that's all that happens; in more modern shells there may be some additional internal bookkeeping to remember state. Understanding it as dup2 means it's easier to understand how successive redirections work, though you also have to know that redirection operators are executed left-to-right, and traditionally each operator was executed immediately as it was parsed, left-to-right. The pipe operator works similarly, though it's a combination of fork and dup'ing, with the command being forked off from the shell as a child before processing the remainder of the line. Though, understanding it this way makes the direction of the angled bracket a little odd; at least for me it's more natural to understand dup2(2, 1) as 2<1, as in make fd 2 a duplicate of fd 1, but in terms of abstract I/O semantics that would be misleading.

emmelaich 8mo ago
Yep, there's a strong unifying feel between the Unix api, C, the shell, and also say Perl. Which is lost when using more modern or languages foreign to Unix.
- tkcranny 8mo ago
  Python too under the hood, a lot of its core is still from how it started as a quick way to do unixy/C things.
ifh-hn 8mo ago
Haha, I'm even more confused now. I have no idea what dup is...
- jpollock 8mo ago
  There are a couple of ways to figure out. open a terminal (OSX/Linux) and type: man dup open a browser window and search for: man dup Both will bring up the man page for the function call. To get recursive, you can try: man man unix (the unix is important, otherwise it gives you manly men)
  - Bender 8mo ago
    otherwise it gives you manly men That's only just after midnight [1][2] [1] - https://www.youtube.com/watch?v=XEjLoHdbVeE https://www.youtube.com/watch?v=XEjLoHdbVeE [2] - https://unix.stackexchange.com/questions/405783/why-does-man-print-gimme-gimme-gimme-at-0030 https://unix.stackexchange.com/questions/405783/why-does-man...
    - ifh-hn 8mo ago
      I love that this situation occured.
  - trashb 8mo ago
    you may also consider gnu info info dup
jez 8mo ago
Another fun consequence of this is that you can initialize otherwise-unset file descriptors this way: $ cat foo.sh #!/usr/bin/env bash >&1 echo "will print on stdout" >&2 echo "will print on stderr" >&3 echo "will print on fd 3" $ ./foo.sh 3>&1 1>/dev/null 2>/dev/null will print on fd 3 It's a trick you can use if you've got a super chatty script or set of scripts, you want to silence or slurp up all of their output, but you still want to allow some mechanism for printing directly to the terminal. The danger is that if you don't open it before running the script, you'll get an error: $ ./foo.sh will print on stdout will print on stderr ./foo.sh: line 5: 3: Bad file descriptor
- 47282847 8mo ago
  Interesting. Is this just literally “fun”, or do you see real world use cases?
  - post-it 8mo ago
    Multiple levels of logging, all of which you want to capture but not all in the same place.
    - skydhash 8mo ago
      Wasn't the idiomatic way the `-v` flag (repeated for verbosity). And then stderr for errors (maybe warning too).
      - notpushkin 8mo ago
        It is, and all logs should ideally go to stderr. But that doesn’t let you pipe them to different places.
      - post-it 8mo ago
        Yes, but sometimes you want just important non-error logs to go to the console or journal, and then those plus verbose logs to go to a file that gets rotated, and then also stderr on top of that.
kccqzy 8mo ago
And just like dup2 allows you to duplicate into a brand new file descriptor, shells also allow you to specify bigger numbers so you aren’t restricted to 1 and 2. This can be useful for things like communication between different parts of the same shell script.
niobe 8mo ago
I find it very intuitive as is
manbash 8mo ago
Respectfully, what was the purpose of this comment, really? And I also disagree, your suggestion is not easier. The & operator is quite intuitive as it is, and conveys the intention.
- goku12 8mo ago
  Perhaps it is intuitive for you based on how you learned it. But their explanation is more intuitive for anyone dealing with low level stuff like POSIX-style embedded programming, low level unix-y C programming, etc, since it ties into what they already know. There is also a limit to how much you can learn about the underlying system and its unseen potential by learning from the abstractions alone. > Respectfully, what was the purpose of this comment, really? Judging by its replies alone, not everyone considers it purposeless. And even though I know enough to use shell redirections correctly, I still found that comment insightful. This is why I still prefer human explanations over AI. It often contains information you didn't think you needed. HN is one of the sources of the gradually dwindling supply of such information. That comment is still on-topic. Please don't discourage such habits.
goku12 8mo ago
This is probably one of the reasons why many find POSIX shell languages to be unpleasant. There are too many syntactical sugars that abstract too much of the underlying mechanisms away, to the level that we don't get it unless someone explains it. Compare this with Lisps, for example. There may be only one branching construct or a looping construct. Yet, they provide more options than regular programming languages using macros. And this fact is not hidden from us. You know that all of them ultimately expand to the limited number of special forms. The shell syntactical sugars also have some weird gotchas. The &2>&1 question and its answer are a good example of that. You're just trading one complexity (low level knowledge) for another (the long list of syntax rules). Shell languages break the rule of not letting abstractions get in the way of insight and intuitiveness. I know that people will argue that shell languages are not programming languages, and that terseness is important for the former. And yet, we still have people complaining about it. This is the programmer ego and the sysadmin ego of people clashing with each other. After all, nobody is purely just one of those two.
- reacweb 8mo ago
  make 2>&1 | tee m.log is in my muscle memory, like adding a & at the end of a command to launch a job, or ctrl+z bg when I forget it, or tar cfz (without the minus so that the order is not important). Without this terseness, people would build myriads of personal alias. This redirection relies on foundational concepts (file descriptors, stdin 0, stdout 1, stderr 2) that need to be well understood when using unix. IMO, this helps to build insight and intuitiveness. A pipe is not magic, it is just a simple operation on file descriptors. Complexity exists (buffering, zombies), but not there.
  - cpach 8mo ago
    Are you sure you understood the comment you replied to? I agree that 2>&1 is not complex. But I think I speak for many Bash users when I say that this idiom looks bad, is hard to Google, hard to read and hard to memorize.
    - skywhopper 8mo ago
jolmg 8mo ago
> Though, understanding it this way makes the direction of the angled bracket a little odd; at least for me it's more natural to understand dup2(2, 1) as 2<1, as in make fd 2 a duplicate of fd 1, but in terms of abstract I/O semantics that would be misleading. Since they're both just `dup2(1, 2)`, `2>&1` and `2<&1` are the same. However, yes, `2<&1` would be misleading because it looks like you're treating stderr like an input.
momentoftop 8mo ago
> The pipe operator works similarly, though it's a combination of fork and dup'ing Any time the shell executes a program it forks, not just for redirections. Redirections will use dup before exec on the child process. Piping will be two forks and obviously the `pipe` syscall, with one process having its stdout dup'd to the input end of the pipe, and the other process having its stdin dup'd to the output end. Honestly, I find the BASH manual to be excellently written, and it's probably available on your system even without an internet connection. I'd always go there than rely on stack overflow or an LLM. https://www.gnu.org/software/bash/manual/bash.html#Redirections https://www.gnu.org/software/bash/manual/bash.html#Redirecti...
ontouchstart 8mo ago
I did a google search on “dup2(2, 1)” in a fresh private tab on my iPhone Safari and this thread came up the second, between https://man7.org/linux/man-pages/man2/dup.2.html https://man7.org/linux/man-pages/man2/dup.2.html and https://man.archlinux.org/man/dup2.2.en https://man.archlinux.org/man/dup2.2.en A lot of bots are reading this. Amazing.

by wahern 8mo ago

emmelaich 8mo ago

Yep, there's a strong unifying feel between the Unix api, C, the shell, and also say Perl. Which is lost when using more modern or languages foreign to Unix.

tkcranny 8mo ago
Python too under the hood, a lot of its core is still from how it started as a quick way to do unixy/C things.

ifh-hn 8mo ago

Haha, I'm even more confused now. I have no idea what dup is...

jpollock 8mo ago
There are a couple of ways to figure out. open a terminal (OSX/Linux) and type: man dup open a browser window and search for: man dup Both will bring up the man page for the function call. To get recursive, you can try: man man unix (the unix is important, otherwise it gives you manly men)
- Bender 8mo ago
  otherwise it gives you manly men That's only just after midnight [1][2] [1] - https://www.youtube.com/watch?v=XEjLoHdbVeE https://www.youtube.com/watch?v=XEjLoHdbVeE [2] - https://unix.stackexchange.com/questions/405783/why-does-man-print-gimme-gimme-gimme-at-0030 https://unix.stackexchange.com/questions/405783/why-does-man...
  - ifh-hn 8mo ago
    I love that this situation occured.
- trashb 8mo ago
  you may also consider gnu info info dup

jez 8mo ago

Another fun consequence of this is that you can initialize otherwise-unset file descriptors this way: $ cat foo.sh #!/usr/bin/env bash >&1 echo "will print on stdout" >&2 echo "will print on stderr" >&3 echo "will print on fd 3" $ ./foo.sh 3>&1 1>/dev/null 2>/dev/null will print on fd 3 It's a trick you can use if you've got a super chatty script or set of scripts, you want to silence or slurp up all of their output, but you still want to allow some mechanism for printing directly to the terminal. The danger is that if you don't open it before running the script, you'll get an error: $ ./foo.sh will print on stdout will print on stderr ./foo.sh: line 5: 3: Bad file descriptor

47282847 8mo ago
Interesting. Is this just literally “fun”, or do you see real world use cases?
- post-it 8mo ago
  Multiple levels of logging, all of which you want to capture but not all in the same place.
  - skydhash 8mo ago
    Wasn't the idiomatic way the `-v` flag (repeated for verbosity). And then stderr for errors (maybe warning too).
    - notpushkin 8mo ago
      It is, and all logs should ideally go to stderr. But that doesn’t let you pipe them to different places.
    - post-it 8mo ago
      Yes, but sometimes you want just important non-error logs to go to the console or journal, and then those plus verbose logs to go to a file that gets rotated, and then also stderr on top of that.

kccqzy 8mo ago

And just like dup2 allows you to duplicate into a brand new file descriptor, shells also allow you to specify bigger numbers so you aren’t restricted to 1 and 2. This can be useful for things like communication between different parts of the same shell script.

niobe 8mo ago

I find it very intuitive as is

manbash 8mo ago

Respectfully, what was the purpose of this comment, really? And I also disagree, your suggestion is not easier. The & operator is quite intuitive as it is, and conveys the intention.

goku12 8mo ago
Perhaps it is intuitive for you based on how you learned it. But their explanation is more intuitive for anyone dealing with low level stuff like POSIX-style embedded programming, low level unix-y C programming, etc, since it ties into what they already know. There is also a limit to how much you can learn about the underlying system and its unseen potential by learning from the abstractions alone. > Respectfully, what was the purpose of this comment, really? Judging by its replies alone, not everyone considers it purposeless. And even though I know enough to use shell redirections correctly, I still found that comment insightful. This is why I still prefer human explanations over AI. It often contains information you didn't think you needed. HN is one of the sources of the gradually dwindling supply of such information. That comment is still on-topic. Please don't discourage such habits.

goku12 8mo ago

This is probably one of the reasons why many find POSIX shell languages to be unpleasant. There are too many syntactical sugars that abstract too much of the underlying mechanisms away, to the level that we don't get it unless someone explains it. Compare this with Lisps, for example. There may be only one branching construct or a looping construct. Yet, they provide more options than regular programming languages using macros. And this fact is not hidden from us. You know that all of them ultimately expand to the limited number of special forms. The shell syntactical sugars also have some weird gotchas. The &2>&1 question and its answer are a good example of that. You're just trading one complexity (low level knowledge) for another (the long list of syntax rules). Shell languages break the rule of not letting abstractions get in the way of insight and intuitiveness. I know that people will argue that shell languages are not programming languages, and that terseness is important for the former. And yet, we still have people complaining about it. This is the programmer ego and the sysadmin ego of people clashing with each other. After all, nobody is purely just one of those two.

reacweb 8mo ago
make 2>&1 | tee m.log is in my muscle memory, like adding a & at the end of a command to launch a job, or ctrl+z bg when I forget it, or tar cfz (without the minus so that the order is not important). Without this terseness, people would build myriads of personal alias. This redirection relies on foundational concepts (file descriptors, stdin 0, stdout 1, stderr 2) that need to be well understood when using unix. IMO, this helps to build insight and intuitiveness. A pipe is not magic, it is just a simple operation on file descriptors. Complexity exists (buffering, zombies), but not there.
- cpach 8mo ago
  Are you sure you understood the comment you replied to? I agree that 2>&1 is not complex. But I think I speak for many Bash users when I say that this idiom looks bad, is hard to Google, hard to read and hard to memorize.
  - skywhopper 8mo ago

jolmg 8mo ago

> Though, understanding it this way makes the direction of the angled bracket a little odd; at least for me it's more natural to understand dup2(2, 1) as 2<1, as in make fd 2 a duplicate of fd 1, but in terms of abstract I/O semantics that would be misleading. Since they're both just `dup2(1, 2)`, `2>&1` and `2<&1` are the same. However, yes, `2<&1` would be misleading because it looks like you're treating stderr like an input.

momentoftop 8mo ago

> The pipe operator works similarly, though it's a combination of fork and dup'ing Any time the shell executes a program it forks, not just for redirections. Redirections will use dup before exec on the child process. Piping will be two forks and obviously the `pipe` syscall, with one process having its stdout dup'd to the input end of the pipe, and the other process having its stdin dup'd to the output end. Honestly, I find the BASH manual to be excellently written, and it's probably available on your system even without an internet connection. I'd always go there than rely on stack overflow or an LLM. https://www.gnu.org/software/bash/manual/bash.html#Redirections https://www.gnu.org/software/bash/manual/bash.html#Redirecti...

ontouchstart 8mo ago

I did a google search on “dup2(2, 1)” in a fresh private tab on my iPhone Safari and this thread came up the second, between https://man7.org/linux/man-pages/man2/dup.2.html https://man7.org/linux/man-pages/man2/dup.2.html and https://man.archlinux.org/man/dup2.2.en https://man.archlinux.org/man/dup2.2.en A lot of bots are reading this. Amazing.