4 ms·

The evolving Unix attitudes on handling signals in your code

77 points by ingve 3y ago

hulitu 3y ago
> The evolving Unix attitudes on handling signals in your code The evolving "Unix attitudes" became non Unix. Bloat is everywhere in Linux.
- sophacles 3y ago
  What do you mean by bloat? I find each person has a different definition. Can you also provide some examples?
  - Blackthorn 3y ago
    Bloat is, of course, anything you aren't using!
    - deleted 3y ago
      [deleted]
- NoZebra120vClip 3y ago
  > The evolving "Unix attitudes" became non Unix. Bloat is everywhere in Linux. I'm not sure what you mean. The point of the article is to explain that older implementations played fast and loose with signal handlers that might do all sorts of operations that turned out to be unsafe. As Chris points out, the current POSIX standard is very constrained! You call some specific functions, you set a flag, you GTFO. So I don't know what you are on about regarding bloat, because if anything in Unix has slimmed down since V7, it's signal handlers (see also the discussion about Bourne shell exploiting SIGSEGV to allocate memory.)
- bitwize 3y ago
  The evolving "Unix attitudes" became non-Unix because the "Unix philosophy" sucks in practice. There really isn't much to it besides "worse is better". Everything in Unix was optimized for ease of implementation rather than ease of developing production applications on top of. The result is a pile of half-solutions: Signals are broken, fork() is broken, traditional Unix synchronous-only I/O is broken. But it was easy for 1970s hackers to build hello-world toy examples on top of those, so nobody really bothered to fix them until relatively recently, and it took brand new APIs largely disjoint from the historical Unix APIs to fix them!
  - convolvatron 3y ago
    I don't think there were 'production applications' in the sense that one means today. there was no gui. no threads. no network. there is nothing wrong with synchronous I/O in a world where programs really only did one thing at a time. signals and fork are pretty bad design decisions. but personally I blame the later generations for not being interested or creative enough to explore alternatives. overall unix was pretty cute for the time.
    - themk 3y ago
      I don't think signals and fork are inherently bad. It was really threads that came along and broke everything. They were poorly thought out and interacted badly with existing features. Perhaps we should have just kept using processes instead.
      - skissane 3y ago
        I think they made a mistake in how they made process-wide signals (so I mean signals like SIGINT not an inherently thread-specific signal such as SIGSEGV) interact with threads. What they should have done with process-wide signals, is have a dedicated thread per each signal handler which does nothing but handle the associated signal (sequentially, blocking it while the handler runs). Yes, it is possible to do this today, but it takes work, and I don’t understand why it wasn’t just the default. I think a handle-based API would have been superior to fork(). So every API which changes shared process state takes a process handle, which could be either the current process or a yet-to-be-started child process. That would have provided most of the advantages of fork() without the many disadvantages.
    - bitwize 3y ago
      Well, not memcached, load-balanced, Kubernetes'd production applications, but computers were certainly running large, multiuser, had to be up with five or more nines of uptime back then. But you're right -- early Unix wasn't running those. Those were handled by IBM iron and, later, VAXen running VMS. Real operating systems. And then college kids who had read the Lions Book as undergrads and thought an operating system was shaped like Unix entered the workforce and decided that VMS and System/370 were too stodgy, and put Unix on those same VAXen. But the operating model of Unix was not designed for real applications with large, concurrent workloads, and the UI was not designed for the people on actual worksites, even in its heyday. (VMS, famously, was designed to be correctly usable assuming the operator is tired and hasn't had their coffee. Unix shells were designed assuming 100% perfect memory and an allergy to actually typing.) I'm sure DonHopkins will be along, endlessly quoting his own contributions to the UNIX-HATERS Handbook, to elucidate.
skissane 3y ago
POSIX could make everyone’s life easier if there was a standard facility to make signals run on dedicated “signal handling threads”. If my SIGINT handler has its own thread that only ever runs that handler (and never runs it re-enterantly), most of these safety issues go away. It is possible to implement that now, but it would be a lot easier if it were just a feature of the C library-and unlike other solutions like the self-pipe trick, the coding model is essentially unchanged (no assumption the program has an event loop). That doesn’t really work for inherently thread-specific signals such as SIGSEGV, but far fewer people are interested in handling those
- gpderetta 3y ago
  That doesn't really work. For a separate thread to be able to tun the signal handler meaningfully, the thread that caused the signal must be stopped. Which would cause the same reentrancy problems. For process-wide signals you can already have dedicated threads.
  - Animats 3y ago
    UNIX signals historically didn't make a distinction between interrupts and exceptions. Some of that distinction has crept in over time, but it's not as explicit as it should be. An exception means the currently executing thread hit a condition that stopped execution, such as a stack overflow, a memory protection violation, a divide by zero, a floating point overflow, bad system call, etc. Those have to be handled by the current thread, which isn't going anywhere until the problem is resolved. Control may be inside locks. You're going to have to abort, unwind, or repair the situation from within that thread. Other signals indicate external events which the program ought to do something about, but don't stop current execution. Those can be handled by any thread, and often are routed to some event-handling thread. Then there's cancellation. If you need to stop a thread, how do you do that? Some operating systems have explicit thread cancellation or inter-thread exceptions. These are the three main cases. Trying to handle all three through one mechanism has resulted in the proliferation of Linux signal handling modes, flags, and functions.
  - skissane 3y ago
RcouF1uZ4gsC 3y ago
Controversial opinion: Unix and C are the two greatest technical debts of all time with respect to computing. We are just now beginning to pay off some of those debts with respect to permissions, isolation, hardware access, instability, memory corruption, and remote code execution. Many of our best practices are more reflective of the mainframes before Unix than Unix.
- voidfunc 3y ago
  I don't think this is all that controversial. Unix and Unix-like systems are basically an incremental series of good enough hacks glued together. It's all quite terrible from a coherent architecture perspective compared to say VMS or it's spiritual successors like WinNT but those also developed in somewhat parallel worlds or are contemporaries of Unix.
- akira2501 3y ago
  POSIX is a standard that offers some amount of backwards compatibility. It's odd to see this as just "technical debt" and entirely ignore what it has accomplished or get anywhere near the question of whether it has been worth it or not. I think, uncontroversially, it clearly has. Particularly, if you look at what the competing "state of the art" has been, I think POSIX has been a major win. If you're willing to ignore the cross platform compatibility it brings you, and write your own implementations, you're suddenly presented with all kinds of nice options for managing signals. For example, signalfd(2) is sweet, and there's no reason to think that some version of this couldn't be standardized by POSIX in the future. Some people see Engineering as a church, where purity is the goal, I see it as a tool, where useful compromise is the goal.
- klodolph 3y ago
  My experience working with Windows is that it is saddled with technical debt more than Posix is. Things like filesystem semantics—you can’t safely put your auxiliary functions in aux.c because most programs will treat that as a device rather than an ordinary file. And the locking mechanisms on Windows are crazy—especially when you're doing something more complicated, like opening an Excel spreadsheet on a CIFS volume. Just a couple illustrations. There are a few things that Windows does better than Unix, but there are a lot of things that Unix does better. The Macintosh basically collapsed under its own technical debt, and the only way Apple got out of it was buying a Posix-like system and writing a compatibility layer for it (and this Posix-like system became OS X). The kernel, xnu, is purportedly a microkernel, but some of the most useful parts of it are the BSD systems that were Frankensteined in.
catiopatio 3y ago
See this related front-page item to see how people still don’t understand the hoops necessary to use these APIs correctly: Using C++23 <stacktrace> to get proper crash logs in C++ programs https://news.ycombinator.com/item?id=36577290 https://news.ycombinator.com/item?id=36577290
schmichael 3y ago
While Go’s signal handling is still not without sharp edges, it is by far the most pleasant system I’ve worked with: signals are handled via channels like any other event in your application. Back when I wrote Python and had to wire up event loops by hand, signalfd seemed to offer a similar experience. Sadly signalfd’s issues are well documented, and it’s really hard to handle signals in complex code bases without help from a runtime like Go’s: https://news.ycombinator.com/item?id=9564975 https://news.ycombinator.com/item?id=9564975
- dllthomas 3y ago
  What are signalfd's issues that Go handles well?
- saagarjha 3y ago
  DispatchSource (from Darwin) is not too bad IMO.
pram 3y ago
MacOS uses Dispatch/GCD to handle signals, the system manages the thread for ObjC/Swift like they described Go and Python. https://developer.apple.com/documentation/dispatch/dispatchsource/2300045-makesignalsource https://developer.apple.com/documentation/dispatch/dispatchs...
- KnobbleMcKnees 3y ago
  Having spent a little time writing toy projects with Go channels, I have an increased appreciation for GCD. I think it has an amazingly low barrier to entry for dealing with simple parallelism. I can see why it's still the gatekeeper for concurrency in iOS and macOS software given how flexible the API is. (In particular, being able to pass queues to publishers in Combine is a little bit of sweetness that makes it more of a joy to use)
xvilka 3y ago
The whole POSIX standard is outdated, especially in terms of safety and security. With all experience and knowledge gained during last two decades, it's time to make a new standard, with better thought APIs, better friendliness to non-C languages (e.g. Rust, C++, etc).
- somat 3y ago
  I think of standards as coming in two flavors, declarative(this is how things should be done) and descriptive(this is how things are currently done). posix is mostly on the descriptive side of things. that being said posix is a prime example of the duality of standards, it is both a terrible standard, encodifying things that probably should not have been encodified and vitally important, it keeps everybody on the same page. On that note I consider posix a good starting point but it is no great sin to abandon it when needed. These days posix is largely "how does linux do it" just like in the past where it was "how does insert_large_unix_vendor_here do it"

77 points by ingve 3y ago

> The evolving Unix attitudes on handling signals in your code The evolving "Unix attitudes" became non Unix. Bloat is everywhere in Linux.

sophacles 3y ago
What do you mean by bloat? I find each person has a different definition. Can you also provide some examples?
- Blackthorn 3y ago
  Bloat is, of course, anything you aren't using!
  - deleted 3y ago
    [deleted]
NoZebra120vClip 3y ago
> The evolving "Unix attitudes" became non Unix. Bloat is everywhere in Linux. I'm not sure what you mean. The point of the article is to explain that older implementations played fast and loose with signal handlers that might do all sorts of operations that turned out to be unsafe. As Chris points out, the current POSIX standard is very constrained! You call some specific functions, you set a flag, you GTFO. So I don't know what you are on about regarding bloat, because if anything in Unix has slimmed down since V7, it's signal handlers (see also the discussion about Bourne shell exploiting SIGSEGV to allocate memory.)
bitwize 3y ago
The evolving "Unix attitudes" became non-Unix because the "Unix philosophy" sucks in practice. There really isn't much to it besides "worse is better". Everything in Unix was optimized for ease of implementation rather than ease of developing production applications on top of. The result is a pile of half-solutions: Signals are broken, fork() is broken, traditional Unix synchronous-only I/O is broken. But it was easy for 1970s hackers to build hello-world toy examples on top of those, so nobody really bothered to fix them until relatively recently, and it took brand new APIs largely disjoint from the historical Unix APIs to fix them!
- convolvatron 3y ago
  I don't think there were 'production applications' in the sense that one means today. there was no gui. no threads. no network. there is nothing wrong with synchronous I/O in a world where programs really only did one thing at a time. signals and fork are pretty bad design decisions. but personally I blame the later generations for not being interested or creative enough to explore alternatives. overall unix was pretty cute for the time.
  - themk 3y ago
    I don't think signals and fork are inherently bad. It was really threads that came along and broke everything. They were poorly thought out and interacted badly with existing features. Perhaps we should have just kept using processes instead.
    - skissane 3y ago
      I think they made a mistake in how they made process-wide signals (so I mean signals like SIGINT not an inherently thread-specific signal such as SIGSEGV) interact with threads. What they should have done with process-wide signals, is have a dedicated thread per each signal handler which does nothing but handle the associated signal (sequentially, blocking it while the handler runs). Yes, it is possible to do this today, but it takes work, and I don’t understand why it wasn’t just the default. I think a handle-based API would have been superior to fork(). So every API which changes shared process state takes a process handle, which could be either the current process or a yet-to-be-started child process. That would have provided most of the advantages of fork() without the many disadvantages.
  - bitwize 3y ago
    Well, not memcached, load-balanced, Kubernetes'd production applications, but computers were certainly running large, multiuser, had to be up with five or more nines of uptime back then. But you're right -- early Unix wasn't running those. Those were handled by IBM iron and, later, VAXen running VMS. Real operating systems. And then college kids who had read the Lions Book as undergrads and thought an operating system was shaped like Unix entered the workforce and decided that VMS and System/370 were too stodgy, and put Unix on those same VAXen. But the operating model of Unix was not designed for real applications with large, concurrent workloads, and the UI was not designed for the people on actual worksites, even in its heyday. (VMS, famously, was designed to be correctly usable assuming the operator is tired and hasn't had their coffee. Unix shells were designed assuming 100% perfect memory and an allergy to actually typing.) I'm sure DonHopkins will be along, endlessly quoting his own contributions to the UNIX-HATERS Handbook, to elucidate.

skissane 3y ago

POSIX could make everyone’s life easier if there was a standard facility to make signals run on dedicated “signal handling threads”. If my SIGINT handler has its own thread that only ever runs that handler (and never runs it re-enterantly), most of these safety issues go away. It is possible to implement that now, but it would be a lot easier if it were just a feature of the C library-and unlike other solutions like the self-pipe trick, the coding model is essentially unchanged (no assumption the program has an event loop). That doesn’t really work for inherently thread-specific signals such as SIGSEGV, but far fewer people are interested in handling those

gpderetta 3y ago
That doesn't really work. For a separate thread to be able to tun the signal handler meaningfully, the thread that caused the signal must be stopped. Which would cause the same reentrancy problems. For process-wide signals you can already have dedicated threads.
- Animats 3y ago
  UNIX signals historically didn't make a distinction between interrupts and exceptions. Some of that distinction has crept in over time, but it's not as explicit as it should be. An exception means the currently executing thread hit a condition that stopped execution, such as a stack overflow, a memory protection violation, a divide by zero, a floating point overflow, bad system call, etc. Those have to be handled by the current thread, which isn't going anywhere until the problem is resolved. Control may be inside locks. You're going to have to abort, unwind, or repair the situation from within that thread. Other signals indicate external events which the program ought to do something about, but don't stop current execution. Those can be handled by any thread, and often are routed to some event-handling thread. Then there's cancellation. If you need to stop a thread, how do you do that? Some operating systems have explicit thread cancellation or inter-thread exceptions. These are the three main cases. Trying to handle all three through one mechanism has resulted in the proliferation of Linux signal handling modes, flags, and functions.
- skissane 3y ago

RcouF1uZ4gsC 3y ago

Controversial opinion: Unix and C are the two greatest technical debts of all time with respect to computing. We are just now beginning to pay off some of those debts with respect to permissions, isolation, hardware access, instability, memory corruption, and remote code execution. Many of our best practices are more reflective of the mainframes before Unix than Unix.

voidfunc 3y ago
I don't think this is all that controversial. Unix and Unix-like systems are basically an incremental series of good enough hacks glued together. It's all quite terrible from a coherent architecture perspective compared to say VMS or it's spiritual successors like WinNT but those also developed in somewhat parallel worlds or are contemporaries of Unix.
akira2501 3y ago
POSIX is a standard that offers some amount of backwards compatibility. It's odd to see this as just "technical debt" and entirely ignore what it has accomplished or get anywhere near the question of whether it has been worth it or not. I think, uncontroversially, it clearly has. Particularly, if you look at what the competing "state of the art" has been, I think POSIX has been a major win. If you're willing to ignore the cross platform compatibility it brings you, and write your own implementations, you're suddenly presented with all kinds of nice options for managing signals. For example, signalfd(2) is sweet, and there's no reason to think that some version of this couldn't be standardized by POSIX in the future. Some people see Engineering as a church, where purity is the goal, I see it as a tool, where useful compromise is the goal.
klodolph 3y ago
My experience working with Windows is that it is saddled with technical debt more than Posix is. Things like filesystem semantics—you can’t safely put your auxiliary functions in aux.c because most programs will treat that as a device rather than an ordinary file. And the locking mechanisms on Windows are crazy—especially when you're doing something more complicated, like opening an Excel spreadsheet on a CIFS volume. Just a couple illustrations. There are a few things that Windows does better than Unix, but there are a lot of things that Unix does better. The Macintosh basically collapsed under its own technical debt, and the only way Apple got out of it was buying a Posix-like system and writing a compatibility layer for it (and this Posix-like system became OS X). The kernel, xnu, is purportedly a microkernel, but some of the most useful parts of it are the BSD systems that were Frankensteined in.

catiopatio 3y ago

See this related front-page item to see how people still don’t understand the hoops necessary to use these APIs correctly: Using C++23 <stacktrace> to get proper crash logs in C++ programs https://news.ycombinator.com/item?id=36577290 https://news.ycombinator.com/item?id=36577290

schmichael 3y ago

While Go’s signal handling is still not without sharp edges, it is by far the most pleasant system I’ve worked with: signals are handled via channels like any other event in your application. Back when I wrote Python and had to wire up event loops by hand, signalfd seemed to offer a similar experience. Sadly signalfd’s issues are well documented, and it’s really hard to handle signals in complex code bases without help from a runtime like Go’s: https://news.ycombinator.com/item?id=9564975 https://news.ycombinator.com/item?id=9564975

dllthomas 3y ago
What are signalfd's issues that Go handles well?
saagarjha 3y ago
DispatchSource (from Darwin) is not too bad IMO.

pram 3y ago

MacOS uses Dispatch/GCD to handle signals, the system manages the thread for ObjC/Swift like they described Go and Python. https://developer.apple.com/documentation/dispatch/dispatchsource/2300045-makesignalsource https://developer.apple.com/documentation/dispatch/dispatchs...

KnobbleMcKnees 3y ago
Having spent a little time writing toy projects with Go channels, I have an increased appreciation for GCD. I think it has an amazingly low barrier to entry for dealing with simple parallelism. I can see why it's still the gatekeeper for concurrency in iOS and macOS software given how flexible the API is. (In particular, being able to pass queues to publishers in Combine is a little bit of sweetness that makes it more of a joy to use)

xvilka 3y ago

The whole POSIX standard is outdated, especially in terms of safety and security. With all experience and knowledge gained during last two decades, it's time to make a new standard, with better thought APIs, better friendliness to non-C languages (e.g. Rust, C++, etc).

somat 3y ago
I think of standards as coming in two flavors, declarative(this is how things should be done) and descriptive(this is how things are currently done). posix is mostly on the descriptive side of things. that being said posix is a prime example of the duality of standards, it is both a terrible standard, encodifying things that probably should not have been encodified and vitally important, it keeps everybody on the same page. On that note I consider posix a good starting point but it is no great sin to abandon it when needed. These days posix is largely "how does linux do it" just like in the past where it was "how does insert_large_unix_vendor_here do it"