6 ms·

The author of beep needs to read the POSIX specification on async-signal-safety [1]. In particular, it is not safe to call exit, free, ioctl, putchar, or perror

The author of beep needs to read the POSIX specification on async-signal-safety [1]. In particular, it is not safe to call exit, free, ioctl, putchar, or perror from a signal handler. [1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2...

madez 9y ago
> In particular, it is not safe to call exit, free, ioctl, putchar, or perror from a signal handler. Why is it accepted by the compiler, then?
- alexbakker 9y ago
  Why wouldn't it be? Why would a compiler know what a signal handler is?
  - mikeash 9y ago
    Because signals and signal handlers are part of the language spec.
    - LukeShu 9y ago
      I was going to disagree and say that they are part of POSIX, not part of C99; but they are indeed part of C99, section 7.14.1 specifically.
      - mikeash 9y ago
        Yeah, it’s a little surprising which parts are where sometimes, and if you spent most of your time on a POSIX system you can easily end up with the wrong assumptions.
- saagarjha 9y ago
  How would the compiler know that a function was a signal handler?
  - alkonaut 9y ago
    It wouldn't. But it might be possible to make it impossible to call forbidden_function() from anywhere where it shouldn't be possible. For example: the signal handler would be a function that accepts an argument of some kind, and the argument is the key to reaching all apis allowed from that point (e.g. the argument must be passed on to allowed apis, or the argument contains function pointers to all allowed functions, that is "OO-style"). I'm not saying this would be possible by any stretch of the imagination for POSIX.
    - alxlaz 9y ago
      How would you differentiate a function that takes an argument of some kind and handles signals from one that just takes an argument of that kind (e.g. for debugging, logging, or some other sort of introspection)? Edit: to be clear -- I'm not asking this just to be snarky :-). Inferring functional information from nothing but semantic information with no functional implication is a very complicated problem that has very error-prone solutions. In C's case, in fact, I think C99 defines signal handlers as taking a single argument, an int. A compiler that doesn't let me call free from a function that takes a single integer argument wouldn't be too useful. IRL, there are analysis tools that can help catch (a subset of) this sort of problem, but they are environment-specific and don't rely on just the function definition. They either look at the signal handler installation calls (but then they're restricted to information that's known at compile time!), or require some sort of annotation (e.g. via special comments a la Doxygen, or via macros etc.). And even then, the sort of problems that they catch are specific to each environment. The restriction isn't that you shouldn't call <these functions> under POSIX, the restriction is that you shouldn't call functions that are not async-signal-safe (i.e. they're not re-entrant or they're not atomic with respect to signals). There are a few POSIX functions that are safe to call from signals (see man 7 signal-safety on a Linux box), but that list obviously doesn't cover user-defined functions -- some of which may be OK to call, others not so much. You can probably determine if a function is async-call-safe with some static analysis but at this point, it's the sort of stuff that really doesn't belong in a compiler anymore...
- jstimpfle 9y ago
  It can't be done in general. A signal handler is just a function that at runtime is connected to a signal. With some work many or most unsafe handlers in the wild can be detected, but so can many other problems, I guess.
  - verytrivial 9y ago
    I'll bite. A helpful compiler would at an appropriate warning level tell you when you are setting a signal handler with a function you haven't (helpfully) annotated with some construct to say "this thing is a signal handler". Said helpful compiler would then also use this helpful annotation to forbid or loudly warn against a helpful subset of naught functions it is able to deduce are being called therein because human programmers cannot be reasonably expected to consume all 64 pages of POSIX function specs before calling any one of them. Seriously.
    - jstimpfle 9y ago
      That's what I was saying. So why don't you step ahead and do it? I'm sure it will be not more than two weeks worth of work. And it will fatten up the compiler only a little bit. And it will annoy proficient programmers only a little bit. How would you deal with calling a function that is externally defined, so you can't easily check if it does something unsafe (or calls another function that does). I'd figure any attempt to check if a signal handler is safe would have to be either extremely expensive or pretty half-arsed. (That's what I was saying, except the bit that programmers cannot reasonably be expected to understand what to do in a signal handler. That you should usually only set a (volatile) flag and return immediately, and should take extreme care if you really need to do more, is basically the first thing you learn when you read up on signal handlers. That's also completely obvious if you take a look at how they are implemented.)
      - verytrivial 9y ago
        Oh, I think we agree. Implementing the feature sounds like a right creeping pain for very little payoff. I would differ wrt "basically the first thing you learn when you read up on signal handlers" -- these days, the crazy kids writing code just start using the language and libraries. Learning why is usually the last thing learnt, and only when something, like a beeping-beep service, sets fire to something.
- Kamshak 9y ago
  This is a really good point actually, aren't there static analysis tools that can figure out things like this? I know in JS we have linters to stop you from doing things that are most likely not very smart. I find this very useful, a lot more useful than to say "If you don't even read the manpages there is no help in sight". Maybe the compiler should not do it but perhaps you could run a linter against packages that are shipped with an os to find issues like this easier.
  - jstimpfle 9y ago
    It's pretty likely that there are tools which can (attempt to) detect this. The question is why aren't these tools used. The answer is because they are expensive, and using them implies development costs as well - learning the tool, setting it up to filter out false positives, setting it up to run in an automated way, structuring the code differently so that the tool is happy (which can lead to worse structure), making sure all the remaining false positives aren't problems every time the tool is run. It's not what you do in an open source project that has a few lines of C code.
  - maxlybbert 9y ago
    C has separate compilation: the compiler isn’t guaranteed to have all of the relevant source code at one time. When the signal handler is compiled, there’s nothing to say “this will be used as a signal handler,” so while the compiler has the source it doesn’t have a reason to complain about calling any particular functions. Then, when that function is used as a signal handler, the compiler has the source to the calling code (well, the code setting up the callback) but may not have the source for the handler itself. So now it knows which rules apply but may not have the code it needs to enforce those rules. If they’re in the same file, it could. And while that is a common case, it’s not the only possibility. How much should the compiler or linter know about the platform’s API? How can I tell the compiler about any arbitrary rule my own code must follow? As someone else suggested, you could do it with annotations, but that’s nonstandard.
  - tejasmanohar 9y ago
    I don't think there's a purely technical reason a non-standard linter cannot exist, but it's probably a large investment to build such a tool (filtering false positives, etc.) and no one has made one sufficiently good and popular enough to be used for a simple one-file-utility.
- simias 9y ago
  I'll be quicker to blame the signal API than the programming language on that front. Dealing with unix signals correctly and robustly is far from trivial and rife with footguns. For instance I believe that Rust still doesn't have a good general purpose solution for handling signals that doesn't involve the libc and unsafe code. Signal is basically the crappiest form of IPC available on a modern operating system short of emulating a mouse and keyboard and typing into the other application's terminal window.
  - DyslexicAtheist 9y ago
    one of my favorite implementation of signal handling has always been DJB's qmail implementation (actually all his code incl djbdns, daemontools etc). His coding style has been a huge inspiration on writing easy to read and secure C.
  - cryptonector 9y ago
    The simplest way to write safe signal handlers is to only ever: - write(2) to STDERR_FILENO - write(2) to a "self-pipe" (i.e., a pipe where the same process is waiting on in its event loop), thus turning the async signal event into an async *I/O* event that can be handled without any constraints regarding async-signal-safety - _exit() Yes, there are other async-signal-safe functions that can be called from a signal handler, but it's generally not worth it. Adhering to my more constrained approach will keep your code safe and will make it easier to always get it right. ALSO, while we're at it, the only global or thread-local variables you can read from or write to from a signal handler must be of type volatile sig_atomic_t (or else volatile of any other integral or pointer type that you can use with atomic operations). This is very important. E.g., imagine using SIGUSR1/2 to manage verbosity levels...
    - bkeroack 9y ago
      Option 2 is very similar to how signal handlers work in Go. When a signal is received, a value is written to a channel and the library user is responsible for reading values from the channel and responding appropriately. https://golang.org/pkg/os/signal/#example_Notify https://golang.org/pkg/os/signal/#example_Notify
- cryptonector 9y ago
  Because the compiler doesn't and cannot know the semantics of signal(2)/sigaction(2). There's what POSIX says, which a compiler could enforce using heuristics, and there's what your C library says. It's perfectly plausible to make a number of nominally not-async-signal-safe functions from the standard actually async-signal-safe in a particular C library. Granted, if you were to take advantage of such an extension, your code would not be portable. (EDIT: The compiler doesn't even know that you're using POSIX. It can figure it out contextually. It doesn't know which C library you'll be linking with though. Bottom line: we need some C standards extensions, or failing that, GCC/Clang C extensions in order to best handle this, though there are some heuristics a compiler could implement even without those, at some mild risk.) (E.g., suppose there was a thread-local counter of signal handlers on the stack... then stdio functions might be able to handle async-signal reentrance. It's not too farfetched, though it's obviously a lot easier to not bother at all and just leave the set of async-signal-safe functions being just the set of system calls that have sufficiently thin stubs in the C library.) Now, the C standard could have additional keywords, much like, say, 'volatile' and friends, to describe async-signal-safety, reentrance, and other characteristics of functions. And if the C library and your programs used these then the compiler absolutely could warn or refuse to compile your program when you break the standard's rules. Incidentally, Unix/POSIX signals are horrible. The only sane and portable way to handle them in programs that do I/O is to have an event loop and a "self-pipe" that the signal handler can write into so that the event loop can pick up and handle the signal as a normal async event in a context where there are no constraints on calling async-signal-unsafe functions. This is what I always do in my programs. I strongly recommend it. This means you don't need ppoll(2), pselect(2), signalfd(2), epoll_pwait(2), and so on -- you don't because you're always turning signals into normal I/O events, so you don't need to worry about signal blocking, and you don't need to use less widely available functions.
- garethrees 9y ago
  This is a good question and doesn't deserve to be downvoted. I think the reason must be that the problem lies in the intersection of three areas of development (the C programming language, the C standard library, and the POSIX operating system definition) and so requires coordination to solve. Think about how you would implement warnings for failures of async-signal-safety. One approach would go like this: 1. In compiler front-ends, introduce a new attribute that can be attached to function declarations to indicate that they are async-signal-safe, for example __attribute__((async_signal_safe)), and a new attribute that can be attached to function parameters and struct members to indicate that the passed value must be async-signal-safe, for example __attribute__((require_async_signal_safe)). 2. In C library headers, apply the async_signal_safe attribute to all the async-signal-safe function declarations, and apply the require_async_signal_safe attribute to the parameter to the signal function and to the sa_handler member of struct sigaction. 3. In compilers, propagate the async_signal_safe attribute, so that any function that only calls async_signal_safe functions is also marked with the attribute. 4. In compilers, check that when a function is passed to a parameter or assigned to a member with the require_async_signal_safe attribute, the function is marked with the async_signal_safe attribute, and issue a warning if not. This doesn't solve the whole problem (sometimes the compiler will not be able to know which function is going to be passed as the parameter to signal, because this is determined at runtime), and it might have false positives in obscure situations (you might in theory use a struct sigaction for some other purpose and never pass it to sigaction) but it would catch many cases, including the one in beep. But notice the amount of coordination required. It would require input from several people with different areas of expertise.
- mikeash 9y ago
  It’s undefined behavior, which means the compiler is free to accept it and do something sensible, or failing subtle ways, or make demons fly out of your nose.
- tinus_hn 9y ago
  A signal handler is not special, it's just another function. So the compiler can't really apply the rules. Also this is a very limited subsection of all the many things you are not supposed to do in a signal handler.

madez 9y ago

> In particular, it is not safe to call exit, free, ioctl, putchar, or perror from a signal handler. Why is it accepted by the compiler, then?

alexbakker 9y ago
Why wouldn't it be? Why would a compiler know what a signal handler is?
- mikeash 9y ago
  Because signals and signal handlers are part of the language spec.
  - LukeShu 9y ago
    I was going to disagree and say that they are part of POSIX, not part of C99; but they are indeed part of C99, section 7.14.1 specifically.
    - mikeash 9y ago
      Yeah, it’s a little surprising which parts are where sometimes, and if you spent most of your time on a POSIX system you can easily end up with the wrong assumptions.
saagarjha 9y ago
How would the compiler know that a function was a signal handler?
- alkonaut 9y ago
  It wouldn't. But it might be possible to make it impossible to call forbidden_function() from anywhere where it shouldn't be possible. For example: the signal handler would be a function that accepts an argument of some kind, and the argument is the key to reaching all apis allowed from that point (e.g. the argument must be passed on to allowed apis, or the argument contains function pointers to all allowed functions, that is "OO-style"). I'm not saying this would be possible by any stretch of the imagination for POSIX.
  - alxlaz 9y ago
    How would you differentiate a function that takes an argument of some kind and handles signals from one that just takes an argument of that kind (e.g. for debugging, logging, or some other sort of introspection)? Edit: to be clear -- I'm not asking this just to be snarky :-). Inferring functional information from nothing but semantic information with no functional implication is a very complicated problem that has very error-prone solutions. In C's case, in fact, I think C99 defines signal handlers as taking a single argument, an int. A compiler that doesn't let me call free from a function that takes a single integer argument wouldn't be too useful. IRL, there are analysis tools that can help catch (a subset of) this sort of problem, but they are environment-specific and don't rely on just the function definition. They either look at the signal handler installation calls (but then they're restricted to information that's known at compile time!), or require some sort of annotation (e.g. via special comments a la Doxygen, or via macros etc.). And even then, the sort of problems that they catch are specific to each environment. The restriction isn't that you shouldn't call <these functions> under POSIX, the restriction is that you shouldn't call functions that are not async-signal-safe (i.e. they're not re-entrant or they're not atomic with respect to signals). There are a few POSIX functions that are safe to call from signals (see man 7 signal-safety on a Linux box), but that list obviously doesn't cover user-defined functions -- some of which may be OK to call, others not so much. You can probably determine if a function is async-call-safe with some static analysis but at this point, it's the sort of stuff that really doesn't belong in a compiler anymore...
jstimpfle 9y ago
It can't be done in general. A signal handler is just a function that at runtime is connected to a signal. With some work many or most unsafe handlers in the wild can be detected, but so can many other problems, I guess.
- verytrivial 9y ago
  I'll bite. A helpful compiler would at an appropriate warning level tell you when you are setting a signal handler with a function you haven't (helpfully) annotated with some construct to say "this thing is a signal handler". Said helpful compiler would then also use this helpful annotation to forbid or loudly warn against a helpful subset of naught functions it is able to deduce are being called therein because human programmers cannot be reasonably expected to consume all 64 pages of POSIX function specs before calling any one of them. Seriously.
  - jstimpfle 9y ago
    That's what I was saying. So why don't you step ahead and do it? I'm sure it will be not more than two weeks worth of work. And it will fatten up the compiler only a little bit. And it will annoy proficient programmers only a little bit. How would you deal with calling a function that is externally defined, so you can't easily check if it does something unsafe (or calls another function that does). I'd figure any attempt to check if a signal handler is safe would have to be either extremely expensive or pretty half-arsed. (That's what I was saying, except the bit that programmers cannot reasonably be expected to understand what to do in a signal handler. That you should usually only set a (volatile) flag and return immediately, and should take extreme care if you really need to do more, is basically the first thing you learn when you read up on signal handlers. That's also completely obvious if you take a look at how they are implemented.)
    - verytrivial 9y ago
      Oh, I think we agree. Implementing the feature sounds like a right creeping pain for very little payoff. I would differ wrt "basically the first thing you learn when you read up on signal handlers" -- these days, the crazy kids writing code just start using the language and libraries. Learning why is usually the last thing learnt, and only when something, like a beeping-beep service, sets fire to something.
Kamshak 9y ago
This is a really good point actually, aren't there static analysis tools that can figure out things like this? I know in JS we have linters to stop you from doing things that are most likely not very smart. I find this very useful, a lot more useful than to say "If you don't even read the manpages there is no help in sight". Maybe the compiler should not do it but perhaps you could run a linter against packages that are shipped with an os to find issues like this easier.
- jstimpfle 9y ago
  It's pretty likely that there are tools which can (attempt to) detect this. The question is why aren't these tools used. The answer is because they are expensive, and using them implies development costs as well - learning the tool, setting it up to filter out false positives, setting it up to run in an automated way, structuring the code differently so that the tool is happy (which can lead to worse structure), making sure all the remaining false positives aren't problems every time the tool is run. It's not what you do in an open source project that has a few lines of C code.
- maxlybbert 9y ago
  C has separate compilation: the compiler isn’t guaranteed to have all of the relevant source code at one time. When the signal handler is compiled, there’s nothing to say “this will be used as a signal handler,” so while the compiler has the source it doesn’t have a reason to complain about calling any particular functions. Then, when that function is used as a signal handler, the compiler has the source to the calling code (well, the code setting up the callback) but may not have the source for the handler itself. So now it knows which rules apply but may not have the code it needs to enforce those rules. If they’re in the same file, it could. And while that is a common case, it’s not the only possibility. How much should the compiler or linter know about the platform’s API? How can I tell the compiler about any arbitrary rule my own code must follow? As someone else suggested, you could do it with annotations, but that’s nonstandard.
- tejasmanohar 9y ago
  I don't think there's a purely technical reason a non-standard linter cannot exist, but it's probably a large investment to build such a tool (filtering false positives, etc.) and no one has made one sufficiently good and popular enough to be used for a simple one-file-utility.
simias 9y ago
I'll be quicker to blame the signal API than the programming language on that front. Dealing with unix signals correctly and robustly is far from trivial and rife with footguns. For instance I believe that Rust still doesn't have a good general purpose solution for handling signals that doesn't involve the libc and unsafe code. Signal is basically the crappiest form of IPC available on a modern operating system short of emulating a mouse and keyboard and typing into the other application's terminal window.
- DyslexicAtheist 9y ago
  one of my favorite implementation of signal handling has always been DJB's qmail implementation (actually all his code incl djbdns, daemontools etc). His coding style has been a huge inspiration on writing easy to read and secure C.
- cryptonector 9y ago
  The simplest way to write safe signal handlers is to only ever: - write(2) to STDERR_FILENO - write(2) to a "self-pipe" (i.e., a pipe where the same process is waiting on in its event loop), thus turning the async signal event into an async *I/O* event that can be handled without any constraints regarding async-signal-safety - _exit() Yes, there are other async-signal-safe functions that can be called from a signal handler, but it's generally not worth it. Adhering to my more constrained approach will keep your code safe and will make it easier to always get it right. ALSO, while we're at it, the only global or thread-local variables you can read from or write to from a signal handler must be of type volatile sig_atomic_t (or else volatile of any other integral or pointer type that you can use with atomic operations). This is very important. E.g., imagine using SIGUSR1/2 to manage verbosity levels...
  - bkeroack 9y ago
    Option 2 is very similar to how signal handlers work in Go. When a signal is received, a value is written to a channel and the library user is responsible for reading values from the channel and responding appropriately. https://golang.org/pkg/os/signal/#example_Notify https://golang.org/pkg/os/signal/#example_Notify
cryptonector 9y ago
Because the compiler doesn't and cannot know the semantics of signal(2)/sigaction(2). There's what POSIX says, which a compiler could enforce using heuristics, and there's what your C library says. It's perfectly plausible to make a number of nominally not-async-signal-safe functions from the standard actually async-signal-safe in a particular C library. Granted, if you were to take advantage of such an extension, your code would not be portable. (EDIT: The compiler doesn't even know that you're using POSIX. It can figure it out contextually. It doesn't know which C library you'll be linking with though. Bottom line: we need some C standards extensions, or failing that, GCC/Clang C extensions in order to best handle this, though there are some heuristics a compiler could implement even without those, at some mild risk.) (E.g., suppose there was a thread-local counter of signal handlers on the stack... then stdio functions might be able to handle async-signal reentrance. It's not too farfetched, though it's obviously a lot easier to not bother at all and just leave the set of async-signal-safe functions being just the set of system calls that have sufficiently thin stubs in the C library.) Now, the C standard could have additional keywords, much like, say, 'volatile' and friends, to describe async-signal-safety, reentrance, and other characteristics of functions. And if the C library and your programs used these then the compiler absolutely could warn or refuse to compile your program when you break the standard's rules. Incidentally, Unix/POSIX signals are horrible. The only sane and portable way to handle them in programs that do I/O is to have an event loop and a "self-pipe" that the signal handler can write into so that the event loop can pick up and handle the signal as a normal async event in a context where there are no constraints on calling async-signal-unsafe functions. This is what I always do in my programs. I strongly recommend it. This means you don't need ppoll(2), pselect(2), signalfd(2), epoll_pwait(2), and so on -- you don't because you're always turning signals into normal I/O events, so you don't need to worry about signal blocking, and you don't need to use less widely available functions.
garethrees 9y ago
This is a good question and doesn't deserve to be downvoted. I think the reason must be that the problem lies in the intersection of three areas of development (the C programming language, the C standard library, and the POSIX operating system definition) and so requires coordination to solve. Think about how you would implement warnings for failures of async-signal-safety. One approach would go like this: 1. In compiler front-ends, introduce a new attribute that can be attached to function declarations to indicate that they are async-signal-safe, for example __attribute__((async_signal_safe)), and a new attribute that can be attached to function parameters and struct members to indicate that the passed value must be async-signal-safe, for example __attribute__((require_async_signal_safe)). 2. In C library headers, apply the async_signal_safe attribute to all the async-signal-safe function declarations, and apply the require_async_signal_safe attribute to the parameter to the signal function and to the sa_handler member of struct sigaction. 3. In compilers, propagate the async_signal_safe attribute, so that any function that only calls async_signal_safe functions is also marked with the attribute. 4. In compilers, check that when a function is passed to a parameter or assigned to a member with the require_async_signal_safe attribute, the function is marked with the async_signal_safe attribute, and issue a warning if not. This doesn't solve the whole problem (sometimes the compiler will not be able to know which function is going to be passed as the parameter to signal, because this is determined at runtime), and it might have false positives in obscure situations (you might in theory use a struct sigaction for some other purpose and never pass it to sigaction) but it would catch many cases, including the one in beep. But notice the amount of coordination required. It would require input from several people with different areas of expertise.
mikeash 9y ago
It’s undefined behavior, which means the compiler is free to accept it and do something sensible, or failing subtle ways, or make demons fly out of your nose.
tinus_hn 9y ago
A signal handler is not special, it's just another function. So the compiler can't really apply the rules. Also this is a very limited subsection of all the many things you are not supposed to do in a signal handler.