5 ms·

It's better to just do nothing and allow SIGPIPE to kill your program. That is the reason this signal exists. Python is not a good unix citizen in this case. Co

It's better to just do nothing and allow SIGPIPE to kill your program. That is the reason this signal exists. Python is not a good unix citizen in this case. Compare it to Perl for example where nothing special is required to do the right thing: $ perl -E 'say "y" while 1' | head -1 y $

v3gas 7y ago
I see
hk__2 7y ago
> Python is not a good unix citizen in this case. Compare it to Perl for example where nothing special is required to do the right thing How is this better? At least with the exception you can cleanly terminate your program if you need to.
- hoytech 7y ago
  In the unix pipeline model, components read input, perform some transformation, and print output. There is no need to do any clean-up. As nneonneo explained, because they rely on SIGPIPE, most unix programs don't even need to check for errors from write(2). Naturally, in the rare case you want to do some clean-up you can: $ perl -E '$SIG{PIPE} = sub { die "cleanup" }; say "y" while 1' | head -1 y cleanup at -e line 1.
seneca 7y ago
I disagree. Saying python is "not a good Unix citizen" here is akin to saying cars are a bad highway citizen because people can crash them. Python provides mechanisms to handle signals. The point of a signal is to indicate something outside of your process has happened and you may want to respond to it. It's up to the program to respond to relevant signals and Python in no way stops a developer from doing that.
- hoytech 7y ago
  > Saying python is "not a good Unix citizen" here is akin to saying cars are a bad highway citizen because people can crash them. I don't follow your analogy. The default signal disposition for SIGPIPE is to silently terminate the process. Normally processes compose nicely on the command-line and this requires no extra work from the programmer. By disregarding this convention, most Python programs pollute your terminal when used in pipelines, which is why I claim that in this case Python is a bad unix citizen.
  - seneca 7y ago
    Yep, I follow your logic. As you say, in other languages it "requires no extra work", it's more a matter of convenience. Python in no way stops you from a having a typical response to SIGPIPE, it's as simple as handling the exception and doing a sys.exit(), but it doesn't "just happen". This makes the typical path a little more contrived, but I don't think having explicit handling makes it a bad citizen. To torture my analogy, in my mind a "bad citizen" of the highway may be a car incapable of doing the minimum speed limit, whereas Python just has a manual transmission vs Perl's automatic (in this case). I'm splitting hairs though, your point is fair. I just think your objection is a little strong for what comes down to convenience.
    - derefr 7y ago
      It's not a matter of "convenience" any more, when you know for a fact that nobody dashing off quick five-line scripts in your language is going to bother doing the extra thing, and so the default behavior will be the only behavior for those scripts. Defaults matter. It's like the difference between opt-in and opt-out for organ donation, or the difference between encouraging and requiring cars to have airbags.
      - lonelappde 7y ago
        Mosst everyone agrees that defaults matter, but not what the defaults should be.
nonofr34 7y ago
Not an expert is no way, so I could be wrong, but in python could you use signal.SIG_DFL to have that expected behavior: # cat.py import sys from signal import signal, SIGPIPE, SIG_DFL signal(SIGPIPE,SIG_DFL) for line in sys.stdin: print(line)
- hoytech 7y ago
  True, but most Python programs don't do this, so they pollute your terminal with error messages when you attempt to use them as interior processes in a unix pipeline.
- gnomewascool 7y ago
  That seems to be discouraged by the official python docs.[0] [0] https://docs.python.org/3/library/signal.html#note-on-sigpipe https://docs.python.org/3/library/signal.html#note-on-sigpip...
  - jolmg 7y ago
    > sys.exit(1) # Python exits with error code 1 on EPIPE Should be 141 instead of 1. Convention is that when a program dies because of a signal, the exit code should be 128 + the signal number, in this case 13. So, 128 + 13 = 141. Here are some examples: $ ( yes; >&2 echo $? ) | head -0 141 $ ( ping localhost; >&2 echo $? ) | head -0 141 $ ( perl -E 'say "y" while 1'; >&2 echo $? ) | head -0 141 In python, when using signal(SIGPIPE, SIG_DFL), you get the correct behaviour (at least regarding the exit status): $ ( python -c ' import sys from signal import signal, SIGPIPE, SIG_DFL signal(SIGPIPE,SIG_DFL) while True: print("y"); ' ; >&2 echo $? ) | head -0 141 > Do not set SIGPIPE’s disposition to SIG_DFL in order to avoid BrokenPipeError. Doing that would cause your program to exit unexpectedly also whenever any socket connection is interrupted while your program is still writing to it. Regarding that socket behavior, if that's the standard in other programming environments, why is it an issue from python's perspective? Doesn't seem worth eschewing standard unix conventions. I mean, it says "unexpectedly", but isn't it actually the expected behavior? Unexpected is for python to say that what's default (SIG_DFL) is unexpected.
dooglius 7y ago
This is bad advice, if you are doing anything other than output to stdout/stderr (which you almost assuredly are, unless you are doing something very simple like "yes"), you want to switch to /dev/null. For instance, running rm -vfr folder/ | head involves SIGPIPE causing problems because it may or may not kill rm before it finishes deleting the directory, based on its internal output buffer size.
- hoytech 7y ago
  The problem here is connecting a program you want to run to completion to the head command, not how rm handles SIGPIPE.
  - dooglius 7y ago
    head is just an example, if you pipe rm -vfr (or any program that does more than just output) into _anything_ the reasonable and default behavior should be to run rm to completion. (And from a programmer's POV, the reasonably thing is for syscalls to handle errors by returning an error, the way every other error is handled.)
    - kees99 7y ago
      Wouldn't it be better to be explicit? If intention is to "display first 10 lines, at most; discard the rest", you can write: rm -vrf directory/ | (head; cat >/dev/null)
      - dooglius 7y ago
        I guess it depends on your semantics/mental model of what a pipeline should do. My mental model says that each part should run fully, and the pipeline should emulate sequential behavior, with "yes" being a bit of a hack in this regard. Cases where the behavior of a pipeline depends on the internal buffer size and operation ordering (e.g. what if rm decided to print everything it did only after removing everything?) seem like bugs to me. Maybe your mental model is different? In any case, I think that in the vast majority of pipelines, the default SIGPIPE behavior is not what is desired.

v3gas 7y ago

I see

hk__2 7y ago

> Python is not a good unix citizen in this case. Compare it to Perl for example where nothing special is required to do the right thing How is this better? At least with the exception you can cleanly terminate your program if you need to.

hoytech 7y ago
In the unix pipeline model, components read input, perform some transformation, and print output. There is no need to do any clean-up. As nneonneo explained, because they rely on SIGPIPE, most unix programs don't even need to check for errors from write(2). Naturally, in the rare case you want to do some clean-up you can: $ perl -E '$SIG{PIPE} = sub { die "cleanup" }; say "y" while 1' | head -1 y cleanup at -e line 1.

seneca 7y ago

I disagree. Saying python is "not a good Unix citizen" here is akin to saying cars are a bad highway citizen because people can crash them. Python provides mechanisms to handle signals. The point of a signal is to indicate something outside of your process has happened and you may want to respond to it. It's up to the program to respond to relevant signals and Python in no way stops a developer from doing that.

hoytech 7y ago
> Saying python is "not a good Unix citizen" here is akin to saying cars are a bad highway citizen because people can crash them. I don't follow your analogy. The default signal disposition for SIGPIPE is to silently terminate the process. Normally processes compose nicely on the command-line and this requires no extra work from the programmer. By disregarding this convention, most Python programs pollute your terminal when used in pipelines, which is why I claim that in this case Python is a bad unix citizen.
- seneca 7y ago
  Yep, I follow your logic. As you say, in other languages it "requires no extra work", it's more a matter of convenience. Python in no way stops you from a having a typical response to SIGPIPE, it's as simple as handling the exception and doing a sys.exit(), but it doesn't "just happen". This makes the typical path a little more contrived, but I don't think having explicit handling makes it a bad citizen. To torture my analogy, in my mind a "bad citizen" of the highway may be a car incapable of doing the minimum speed limit, whereas Python just has a manual transmission vs Perl's automatic (in this case). I'm splitting hairs though, your point is fair. I just think your objection is a little strong for what comes down to convenience.
  - derefr 7y ago
    It's not a matter of "convenience" any more, when you know for a fact that nobody dashing off quick five-line scripts in your language is going to bother doing the extra thing, and so the default behavior will be the only behavior for those scripts. Defaults matter. It's like the difference between opt-in and opt-out for organ donation, or the difference between encouraging and requiring cars to have airbags.
    - lonelappde 7y ago
      Mosst everyone agrees that defaults matter, but not what the defaults should be.

nonofr34 7y ago

Not an expert is no way, so I could be wrong, but in python could you use signal.SIG_DFL to have that expected behavior: # cat.py import sys from signal import signal, SIGPIPE, SIG_DFL signal(SIGPIPE,SIG_DFL) for line in sys.stdin: print(line)

hoytech 7y ago
True, but most Python programs don't do this, so they pollute your terminal with error messages when you attempt to use them as interior processes in a unix pipeline.
gnomewascool 7y ago
That seems to be discouraged by the official python docs.[0] [0] https://docs.python.org/3/library/signal.html#note-on-sigpipe https://docs.python.org/3/library/signal.html#note-on-sigpip...
- jolmg 7y ago
  > sys.exit(1) # Python exits with error code 1 on EPIPE Should be 141 instead of 1. Convention is that when a program dies because of a signal, the exit code should be 128 + the signal number, in this case 13. So, 128 + 13 = 141. Here are some examples: $ ( yes; >&2 echo $? ) | head -0 141 $ ( ping localhost; >&2 echo $? ) | head -0 141 $ ( perl -E 'say "y" while 1'; >&2 echo $? ) | head -0 141 In python, when using signal(SIGPIPE, SIG_DFL), you get the correct behaviour (at least regarding the exit status): $ ( python -c ' import sys from signal import signal, SIGPIPE, SIG_DFL signal(SIGPIPE,SIG_DFL) while True: print("y"); ' ; >&2 echo $? ) | head -0 141 > Do not set SIGPIPE’s disposition to SIG_DFL in order to avoid BrokenPipeError. Doing that would cause your program to exit unexpectedly also whenever any socket connection is interrupted while your program is still writing to it. Regarding that socket behavior, if that's the standard in other programming environments, why is it an issue from python's perspective? Doesn't seem worth eschewing standard unix conventions. I mean, it says "unexpectedly", but isn't it actually the expected behavior? Unexpected is for python to say that what's default (SIG_DFL) is unexpected.

dooglius 7y ago

This is bad advice, if you are doing anything other than output to stdout/stderr (which you almost assuredly are, unless you are doing something very simple like "yes"), you want to switch to /dev/null. For instance, running rm -vfr folder/ | head involves SIGPIPE causing problems because it may or may not kill rm before it finishes deleting the directory, based on its internal output buffer size.

hoytech 7y ago
The problem here is connecting a program you want to run to completion to the head command, not how rm handles SIGPIPE.
- dooglius 7y ago
  head is just an example, if you pipe rm -vfr (or any program that does more than just output) into _anything_ the reasonable and default behavior should be to run rm to completion. (And from a programmer's POV, the reasonably thing is for syscalls to handle errors by returning an error, the way every other error is handled.)
  - kees99 7y ago
    Wouldn't it be better to be explicit? If intention is to "display first 10 lines, at most; discard the rest", you can write: rm -vrf directory/ | (head; cat >/dev/null)
    - dooglius 7y ago
      I guess it depends on your semantics/mental model of what a pipeline should do. My mental model says that each part should run fully, and the pipeline should emulate sequential behavior, with "yes" being a bit of a hack in this regard. Cases where the behavior of a pipeline depends on the internal buffer size and operation ordering (e.g. what if rm decided to print everything it did only after removing everything?) seem like bugs to me. Maybe your mental model is different? In any case, I think that in the vast majority of pipelines, the default SIGPIPE behavior is not what is desired.