4 ms·

This is a good article, but this bit irks me: $ printf "Look, I have something inside" > /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something i

by jolmg 8y ago

This is a good article, but this bit irks me: $ printf "Look, I have something inside" > /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something inside $ printf "\nNow I have one more thing" >> /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something inside Now I have one more thing $ What really happens is: $ printf "Look, I have something inside" > /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something inside$ printf "\nNow I have one more thing" >> /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something inside Now I have one more thing$ And what you want is: $ printf "Look, I have something inside\n" > /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something inside $ printf "Now I have one more thing\n" >> /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something inside Now I have one more thing $ Newlines are line terminators, not separators. This article missed what could have been a really good section on line buffering that could have explained some of the merit of having line terminators as opposed to line separators.

klodolph 8y ago
One of the reasons that I like zsh, $ printf "Look, I have something inside" > /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something inside% $ printf "\nNow I have one more thing" >> /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something inside Now I have one more thing% $ Zsh will add the line break and mark its absence with a % with reverse video). I follow the "terminator-not-separator" rule religiously in my own source code, but my impression is that this is somewhat a Unix-ism.
- saagarjha 8y ago
  > my impression is that this is somewhat a Unix-ism This is how POSIX defines lines for text files.
  - dfox 8y ago
    Which defines it this way, because various original Unix components (including things like stdio) will break in various surprising and inconsistent ways when text file does not end in \n or contains lines longer than some arbitrary limit (which is why POSIX has LINE_MAX).
    - jolmg 8y ago
      It's very much in favour of consistency as well as simplicity and therefore predictability that newlines should be terminators. What would you have `cat file1 file2` do? Add newlines in between the files? It simply concatenates. It shouldn't even differentiate on the content. It's just arbitrary binary data to it. I can even use it on mp3s to join them. Why should it be limited to text files with newline separators? Add an option to add newlines in between the files? Why complicate matters? Simplicity is king! Let's do more with less software.
- jpeeler 8y ago
  Fish will also show ⏎ when missing a newline control character (or characters on other platforms too I guess).
- jolmg 8y ago
  First of all, that's a hack zsh really shouldn't have to do. It's not like it can see the output of the commands that it runs. That's the job of the terminal. To make that '%' effect work, when a command it's waiting for dies, it outputs a '%' and then as many spaces as it knows there are columns in the terminal and then a carriage return '\r'. So it uses line-wrapping to move to the next line if there's content in the current one. That's really out of scope of what a shell should be doing. It also will not work if you're not working with the shell through a terminal, but something else like a tcp connection or a serial device. What's going to happen then is that that last "line" is going to disappear in zsh. The prompt will be rendered on top of it. Anyway, it's not just about the shell prompt. Things are just simpler and more composable when using it as a terminator instead of a separator. Consider programs that might every now and then have to warn you about errors, like this: cmd() { >&2 printf "%s-error-1" "$1" sleep 1 >&2 printf "\n%s-error-2" "$1" } and them being used in pipe: cmd foo | cmd bar using newlines as separators causes them to merge lines like so: foo-error-1bar-error-1 foo-error-2 bar-error-2% if one used line terminators: cmd() { >&2 printf "%s-error-1\n" "$1" sleep 1 >&2 printf "%s-error-2\n" "$1" } then the output would be like so: foo-error-1 bar-error-1 foo-error-2 bar-error-2 Now consider what would happen if you need to conditionalize the first line: cmd() { if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-1" "$1" fi sleep 1 >&2 printf "\n%s-error-2" "$1" } Now, if that first line doesn't output anything, you'll get a extra, blank line. So, you need to include the newline in the conditional body: cmd() { if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-1" "$1" >&2 printf "\n" fi sleep 1 >&2 printf "%s-error-2" "$1" } Now what if you need to conditionalize the last line? cmd() { if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-1" "$1" >&2 printf "\n" fi sleep 1 if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-2" "$1" end } should the newline go in the first or second conditional body? If it's on the first, then if the first condition is true and the second is false you'll get a trailing newline (something you don't want if you're trying to have separators). What's the solution here? The newline will have to be added only when BOTH conditional bodies run: cmd() { if (( $RANDOM % 2 == 0 )); then first_was_true=1 >&2 printf "%s-error-1" "$1" fi sleep 1 if (( $RANDOM % 2 == 0 )); then if [[ $first_was_true ]]; then >&2 printf "\n" fi >&2 printf "%s-error-2" "$1" end } What if you have a 3rd conditional error? Well, that's a new newline that will have to be output iff the second and third ran or the first and third ran: cmd() { if (( $RANDOM % 2 == 0 )); then first_was_true=1 >&2 printf "%s-error-1" "$1" fi sleep 1 if (( $RANDOM % 2 == 0 )); then second_was_true=1 if [[ $first_was_true ]]; then >&2 printf "\n" fi >&2 printf "%s-error-2" "$1" end sleep 1 if (( $RANDOM % 2 == 0 )); then if [[ $first_was_true || $second_was_true ]]; then >&2 printf "\n" fi >&2 printf "%s-error-2" "$1" end } Can you see how this quickly becomes unwieldy? Each new body becomes coupled to the ones before it. The third conditional needs to know about the first and second when semantically it shouldn't need to care at all! If you use line terminators, things are much simpler: cmd() { if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-1\n" "$1" fi sleep 1 if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-2\n" "$1" fi sleep 1 if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-3\n" "$1" fi } This isn't just an arbitrary Unix decision. It's objectively good design.

by jolmg 8y ago

klodolph 8y ago

One of the reasons that I like zsh, $ printf "Look, I have something inside" > /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something inside% $ printf "\nNow I have one more thing" >> /tmp/careful.txt $ cat /tmp/careful.txt Look, I have something inside Now I have one more thing% $ Zsh will add the line break and mark its absence with a % with reverse video). I follow the "terminator-not-separator" rule religiously in my own source code, but my impression is that this is somewhat a Unix-ism.

saagarjha 8y ago
> my impression is that this is somewhat a Unix-ism This is how POSIX defines lines for text files.
- dfox 8y ago
  Which defines it this way, because various original Unix components (including things like stdio) will break in various surprising and inconsistent ways when text file does not end in \n or contains lines longer than some arbitrary limit (which is why POSIX has LINE_MAX).
  - jolmg 8y ago
    It's very much in favour of consistency as well as simplicity and therefore predictability that newlines should be terminators. What would you have `cat file1 file2` do? Add newlines in between the files? It simply concatenates. It shouldn't even differentiate on the content. It's just arbitrary binary data to it. I can even use it on mp3s to join them. Why should it be limited to text files with newline separators? Add an option to add newlines in between the files? Why complicate matters? Simplicity is king! Let's do more with less software.
jpeeler 8y ago
Fish will also show ⏎ when missing a newline control character (or characters on other platforms too I guess).
jolmg 8y ago
First of all, that's a hack zsh really shouldn't have to do. It's not like it can see the output of the commands that it runs. That's the job of the terminal. To make that '%' effect work, when a command it's waiting for dies, it outputs a '%' and then as many spaces as it knows there are columns in the terminal and then a carriage return '\r'. So it uses line-wrapping to move to the next line if there's content in the current one. That's really out of scope of what a shell should be doing. It also will not work if you're not working with the shell through a terminal, but something else like a tcp connection or a serial device. What's going to happen then is that that last "line" is going to disappear in zsh. The prompt will be rendered on top of it. Anyway, it's not just about the shell prompt. Things are just simpler and more composable when using it as a terminator instead of a separator. Consider programs that might every now and then have to warn you about errors, like this: cmd() { >&2 printf "%s-error-1" "$1" sleep 1 >&2 printf "\n%s-error-2" "$1" } and them being used in pipe: cmd foo | cmd bar using newlines as separators causes them to merge lines like so: foo-error-1bar-error-1 foo-error-2 bar-error-2% if one used line terminators: cmd() { >&2 printf "%s-error-1\n" "$1" sleep 1 >&2 printf "%s-error-2\n" "$1" } then the output would be like so: foo-error-1 bar-error-1 foo-error-2 bar-error-2 Now consider what would happen if you need to conditionalize the first line: cmd() { if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-1" "$1" fi sleep 1 >&2 printf "\n%s-error-2" "$1" } Now, if that first line doesn't output anything, you'll get a extra, blank line. So, you need to include the newline in the conditional body: cmd() { if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-1" "$1" >&2 printf "\n" fi sleep 1 >&2 printf "%s-error-2" "$1" } Now what if you need to conditionalize the last line? cmd() { if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-1" "$1" >&2 printf "\n" fi sleep 1 if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-2" "$1" end } should the newline go in the first or second conditional body? If it's on the first, then if the first condition is true and the second is false you'll get a trailing newline (something you don't want if you're trying to have separators). What's the solution here? The newline will have to be added only when BOTH conditional bodies run: cmd() { if (( $RANDOM % 2 == 0 )); then first_was_true=1 >&2 printf "%s-error-1" "$1" fi sleep 1 if (( $RANDOM % 2 == 0 )); then if [[ $first_was_true ]]; then >&2 printf "\n" fi >&2 printf "%s-error-2" "$1" end } What if you have a 3rd conditional error? Well, that's a new newline that will have to be output iff the second and third ran or the first and third ran: cmd() { if (( $RANDOM % 2 == 0 )); then first_was_true=1 >&2 printf "%s-error-1" "$1" fi sleep 1 if (( $RANDOM % 2 == 0 )); then second_was_true=1 if [[ $first_was_true ]]; then >&2 printf "\n" fi >&2 printf "%s-error-2" "$1" end sleep 1 if (( $RANDOM % 2 == 0 )); then if [[ $first_was_true || $second_was_true ]]; then >&2 printf "\n" fi >&2 printf "%s-error-2" "$1" end } Can you see how this quickly becomes unwieldy? Each new body becomes coupled to the ones before it. The third conditional needs to know about the first and second when semantically it shouldn't need to care at all! If you use line terminators, things are much simpler: cmd() { if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-1\n" "$1" fi sleep 1 if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-2\n" "$1" fi sleep 1 if (( $RANDOM % 2 == 0 )); then >&2 printf "%s-error-3\n" "$1" fi } This isn't just an arbitrary Unix decision. It's objectively good design.