18 ms·

fork() can fail

760 points by dantiberian 12y ago

jgrahamc 12y ago
Quietly goes to check the last piece of C I wrote containing a fork(): if (daemon && !test_mode) { int pid = fork(); if (pid == -1) { fatal_error("Failed to fork"); } if (pid != 0) { write_pid(pid_file, pid, !test_mode); exit(0); } } else { write_pid(pid_file, getpid(), !test_mode); } Phew!
- cperciva 12y ago
  I usually use switch with fork: if (daemon && !test_mode) { int pid; switch (pid = fork()) { case -1: /* Error */ fatal_error("Failed to fork"); case 0: /* In child */ break; default: /* In parent */ write_pid(pid_file, pid, !test_mode); exit(0); } } else { write_pid(pid_file, getpid(), !test_mode); }
  - jgrahamc 12y ago
    Yes. That's rather nice. Although I'm not a fan of the (pid = fork()) inline assignment and condition. But that's a matter of taste, not technology.
    - cperciva 12y ago
      Sure, you could have pid = fork(); switch (pid) if you prefer. I find that the inline assignment-and-condition style is clearer since it reads to me as "switch on the result of fork, and cache that value somewhere" whereas the separate statements read to me as "call fork" and "switch on the process ID".
  - colanderman 12y ago
    For daemonization, daemon(3) is better (EDIT: assuming you only care about Linux). (It also chdirs to /, closes STD*, and detaches from the terminal.)
    - cperciva 12y ago
      Alas, daemon(3) is not POSIX.
    - mindajar 12y ago
      daemon(3) isn't part of POSIX, and the OS X man page says "the use of this API is discouraged in favor of using launchd(8)", so who knows what might happen here in the future. If you don't care about portability, it's an easier call to make.
  - cliveowen 12y ago
    You should always put a break statement at the end of a default branch. The default case is put at the end as a convention but no one (certainly not the C standard) prevents you from adding a new case after, which will be promptly executed even though probably it's what you want. I think this is mentioned a best practice in the K&R.
    - cperciva 12y ago
      You mean, after exit(0) returns?
      - AnimalMuppet 12y ago
        Not after exit(0) returns, no. Instead, you're going to want it after some programmer removes exit(0) six months from now.
        loup-vaillant 12y ago
        I'm going to write a pre-processor which adds the appropriate `break` statements automatically. No more fall-through bug. (And if you really want to fall through, I could add a `fallthrough` keyword.)
        cperciva 12y ago
        For what it's worth, it's a style bug in FreeBSD to have a fallthrough which does not have a /* FALLTHROUGH */ comment.
        scintill76 12y ago
        Yep, could even re-use the "continue" keyword.
        hamburglar 12y ago
        smart-ass comment: you should put two breaks in for when some programmer removes one of them six months from now. ;) edit: i know it's not totally analogous since removing the exit and not noticing there's no break is a lot more likely than just randomly removing a break, but the "let's prevent someone clumsy from screwing this code up in the future" argument always makes me laugh a little.
  - rosser 12y ago
    I'm kinda surprised that the assignment-during-test thing doesn't generate a warning in the case of a switch statement. It does in other cases (if and while do under gcc and clang, at least).
    - cperciva 12y ago
      If it creates warnings, you can silence them by adding extra parentheses, e.g., if ((buf = malloc(buflen)) == NULL) goto outofmemory;
      - rosser 12y ago
        I was originally going to comment saying exactly that — under gcc and clang, at least, just the extra parens will disable the warning; no comparison necessary — but thought to test that the warning is generated in case of a switch statement (and is then disabled by the extra set of parens), which led to realizing the behavior is different. Just surprised by the inconsistency.
      - loup-vaillant 12y ago
        Careful with that example. The parentheses here don't just remove the warning. They remove a bug. This code: if (buf = malloc(buflen) == NULL) goto outofmemory; is actually equivalent to that code: if (buf = (malloc(buflen) == NULL)) goto outofmemory; So, malloc gives you a pointer, which is compared to the null pointer, giving you either 0 or 1. And that is assigned to buf. Hopefully your compiler will warn you about the type error that spawns from such dark magic.
        deleted 12y ago
        [deleted]
        cperciva 12y ago
        Oops, quite right. I was originally thinking of if ((rc = pthread_mutex_lock(mtx))) err("mutex_lock failed: %s", strerror(rc)); but decided to switch to a better-known function at the last minute and completely lost the point.
  - ChuckMcM 12y ago
    Somewhat pedantically, I discovered a bug where a daemon (which did this very similar code) started killing random processes. The issue turned out to be that it was test run as root once and the pid_file was owned by root, group root. So the write_pid() function failed (silently) and the cleanup script always took the pid that was in pid_file and sent it a kill 9, which was now stuck. Sometimes kill 9 would return invalid pid, sometimes it would kill some process. Fixed by removing the pid_file, letting the daemon create it, and checking that the write succeeded.
- jervisfm 12y ago
  I was curious too about my own code, so I looked at the last publicly available C code I wrote with fork in it[1], and yep I checked for the error case too :) I think this just came from being drilled in school on the importance of checking error return values. No matter how unlikely never assume that something can't happen. If it really can't you should at the very least assert on it. [1] - https://github.com/jervisfm/W4118_HW1/blob/master/shell/shell.c#L300 https://github.com/jervisfm/W4118_HW1/blob/master/shell/shel...
cperciva 12y ago
This reminds me of one of the most epic bugs I've ever run into: mkdir("/foo", 0700); chdir("/foo"); recursively_delete_everything_in_current_directory(); Running as root, this usually worked fine: It would create a directory, move into it, and clean out any garbage left behind by a previous run before doing anything new. Running as non-root, the mkdir failed, the chdir failed, and it started eating my home directory.
- stinos 12y ago
  When you see chdir, or any notion of the current working directory being used for anything: run as fast as you can. (or refactor if it's not too late). Things I've seen because of software relying on it.. Sometimes it's just directories/files it creates popping up all over the place, sometimes it's 'just' crashing, but yes sometimes it starts to erase and all hell really breaks loose.
  - cperciva 12y ago
    In this particular case, I didn't see the chdir until after the fact. This wasn't my code...
  - dllthomas 12y ago
    Right, better to use the *at() functions (e.g. http://linux.die.net/man/2/unlinkat http://linux.die.net/man/2/unlinkat) where applicable.
  - agwa 12y ago
    Sometimes there are legitimate reasons for relying on the CWD, such as avoiding time-of-check/time-of-use race conditions. The *at syscalls provide a better alternative, but (at least as of a few years ago) they weren't widely implemented outside of Linux and Solaris.
  - pachydermic 12y ago
    If you can't rely on current working directories then you have to specify any file locations absolutely? That doesn't seem like a good idea because then your code quickly turns into a hot mess if you ever have to change where stuff lives. This is such a stupid problem I run into a lot. Both alternatives (doing things with absolute paths vs doing things entirely with relative paths) seem to have a lot of downsides. Overall relative paths seems to be way better, but then you leave yourself open to problems which "rhyme" with the one OP was talking about.
runarb 12y ago
Easy to test out to. In C: #include <unistd.h> int main(void) { while(1) { fork(); } }
- tdicola 12y ago
  Haha too cruel. I remember in one of my early college CS classes the professor told us about fork bombs and wasn't sure if they still worked on the Sun Fire servers we were using. About 5 minutes later someone piped up and confirmed that yep they still worked at bringing the server down. :)
  - X-Istence 12y ago
    That's why user accounts should have appropriate resource limits set per user.
- arthurcolle 12y ago
  while(fork()) {};
  - TheLoneWolfling 12y ago
    while(1) {fork()}; is "better", as the version you mention will quit as soon as it is able to fork a new process.
    - jimm 12y ago
      Will it quit? A failed fork() will return -1, which is non-zero thus true in a boolean context. The while(fork()) {...} will run.
      - comex 12y ago
        But it will stop when fork returns 0, i.e. in the child processes, making it non-exponential.
    - 12y ago
Aurel1us 12y ago
Just as a reminder: "So, malloc on Linux only fails if there isn’t enough memory for its control structures. It does not fail if there isn’t enough memory to fulfill the request." - http://scvalex.net/posts/6/ http://scvalex.net/posts/6/
- stonogo 12y ago
  This is a mostly-untrue statement because it makes unreliable assumptions about the host system. It depends on the vm.overcommit_memory setting and the programmer should never make assumptions about why or when malloc might fail. Read more on Rich Felker's excellent blog post here: http://ewontfix.com/3/ http://ewontfix.com/3/
- acdha 12y ago
  That page is incomplete: malloc() will fail on Linux depending on the `vm_overcommit` setting: https://www.kernel.org/doc/Documentation/vm/overcommit-accounting https://www.kernel.org/doc/Documentation/vm/overcommit-accou...
- quotemstr 12y ago
  Malloc can also fail if you're out of address space without necessarily being out of memory. NT does a much better job of separating these concepts than Unix-family operating systems do. Conceptually, setting aside a region of your process's address space and guaranteeing that the OS will be able to serve you a given number of pages are completely different operations. I wish more programs would use MAP_NORESERVE when they want the former without the latter. (I'm looking at you, Java.) One day, perhaps when I am old and frail, we will achieve sanity and turn overcommit off by default. But we're a long way from being able to do that now.
  - klodolph 12y ago
    These days, I describe malloc() as "a function which allocates address space", to avoid confusion. Which means it makes sense that malloc() returns NULL if you are out of address space, even if you have lots of memory. (But so many people don't check malloc()'s return anyway...)
zokier 12y ago
Who needs type safety when we got integers.
- zxcdw 12y ago
  Who even wants that? When I am coding, sure as hell some compiler vendor won't be telling me what to use my bits for! I identify myself as an atypical coder.
  - kazinator 12y ago
    If you tell yourself what the bits are for and stick to it, that is still type. Real chaos is when even you don't.
    - deleted 12y ago
      [deleted]
- AnimalMuppet 12y ago
  Is this just using some vague connection to ride on one of your favorite hobby horses? Or does this have a connection to the article, and I missed it?
  - GhotiFish 12y ago
    well you're no fun.
  - mkehrt 12y ago
    The problem is that C and POSXIX don't (idiomatically) provide rich enough data types force checking the error condition. The fact that the error is signaled by a random integer (-1) is horrible. In a language with stronger types and richer data structures, one can have a return type that is a disjunction of {failure, parent, child}, so you can never accidentally treat a failure as a PID. In a functional language this would look like a datatype (essentially a generalized enum), while an OO language you would use different subclasses of a common superclass. In C you can return a tagged union and check the tag, but nothing forces you to do the check. A user of this API can just go ahead and assume the success branch of the union. Furthermore, this isn't idiomatic POSIX, so it is never done. [edit: "subclass" -> "superclass"]
emo_tards_on_hn 12y ago
This is an important point, along similar lines to something we discussed at work today in a code review. The programmer in question assumed that all and every request for the resources consumed would always work. Literally, no testing or consideration for what if one of those request fails. Will be emailing them this article tomorrow!
- arbitrage 12y ago
  Your username is offensive. I'd like to take what you say seriously, and perhaps even engage in further conversation about the topic at hand, but ... you know ... you look stupid. Grow up. You should be embarrassed.
  - dang 12y ago
    That's understandable, but please don't feed the trolls. When you see a comment that is truly bad for HN, it's best to flag it. You can do this by clicking on "link" to go to the item page for the comment, then "flag". We monitor those flags and take action based on them.
  - stinos 12y ago
    Shouldn't take long user: emo_tards_on_hn created: 48 minutes ago karma: -20
  - slingerofwheat 12y ago
    So. You mean you actually DON'T like to take what he said seriously. His username is offensive, but you just also gave it credibility.
    - schrodinger 12y ago
      "I'd like to" means I would like to, presumably meaning "I would like to take your post seriously, but your offensive handle is preventing me"
japaget 12y ago
See also https://news.ycombinator.com/item?id=8189968 https://news.ycombinator.com/item?id=8189968 for another UNIX trap for the unwary.
- mnw21cam 12y ago
  I sometimes drop a "--help" or "--version" file in a directory I don't want to accidentally run "rm *" in.
dasmithii 12y ago
I'm uncertain that I've ever checked for errors after calling fork. Thanks for providing an impetus to do so.
mutation 12y ago
Just noticed that in Perl the behavior is slightly different: http://perldoc.perl.org/functions/fork.html http://perldoc.perl.org/functions/fork.html unsuccessful fork() returns undef, effectively stopping you from kill-ing what you don't want to kill.
- epochwolf 12y ago
  In C, there is no concept of null or undefined, only 0.
  - dllthomas 12y ago
    That's not quite true, but not a terrible approximation.
    - sdegutis 12y ago
      How is it not true? Besides void expressions (which aren't values) the above poster's statement appears correct.
      - dllthomas 12y ago
        A null pointer does not need to have an all-zero representation, which mattered once on some architectures. I'm not aware of any current architecture that does it that way, but that's still what the standard says, which is to say that C does technically have a notion of null which is not necessarily identical with 0. http://c-faq.com/null/machexamp.html http://c-faq.com/null/machexamp.html
        spc476 12y ago
        But in C, a 0 in a pointer context is always considered the NULL pointer (even if the physical value of a NULL pointer isn't all zeros).
        loup-vaillant 12y ago
spudlyo 12y ago
If a function be advertised to return an error code in the event of difficulties, thou shalt check for that code, yea, even though the checks triple the size of thy code and produce aches in thy typing fingers, for if thou thinkest "it cannot happen to me", the gods shall surely punish thee for thy arrogance. [0] [0]: http://www.lysator.liu.se/c/ten-commandments.html http://www.lysator.liu.se/c/ten-commandments.html
- leni536 12y ago
  Note that wasn't the real reason. At the bottom you can see an edit which reads: "I was wrong about why malloc finally failed! @GodmarBack observes, in the comments, that x64 systems only have an address space of 48 bits, which comes out to about 131000 GB. So, on my machine at least, the malloc finally failed because of address space exhaustion."
  - eshyong 12y ago
    FYI, I think you replied to the wrong message :)
- quotemstr 12y ago
  Counterexample: pthread_mutex_unlock. That function returns an error code, but it cannot possibly fail in a well-formed program. Checking for an error for mutex unlock is pointless: what would you do in response?
  - cpeterso 12y ago
    > cannot possibly fail in a well-formed program I think that's your answer. A mutex error return probably indicates an application bug, such as double unlock. You should probably assert or abort on "can't happen" mutex errors. Programmers are lazy. If they take the time to document an error return value, then you should probably heed their warnings. :)
    - quotemstr 12y ago
VLM 12y ago
"Neither of them fail often" See /etc/security/limits.conf and nproc and "fork bomb" Aside from intentional fork bombs I've seen this done intentionally in the spirit of a OOMkiller to keep a machine alive for debugging / detection of problem. 100 "whatever" processes will kill this webserver making it impossible to log in and diagnose much less fix, so we'll limit to 50 processes in the OS. I've also seen it in systems where people are too lazy to test if a process is running before forking another and the system doesn't like multiple copies running (like a keep alive restarter pattern). If ops has no access to the source to fix that or no one cares, then just run it in jail where you only get two processes, the restarter-forker and the forkee. Then hilarity can result if the restarter thinks the PID of the failed fork means something, like sending an email alert or logging the restart attempt. "Why are my logs now gigabytes of ERROR: restarted process new pid is -1?"
walski 12y ago
Thanks! Definitively in my "shit I should know but didn't before HN schooled me"-top-10 :)
- vigneshv_psg 12y ago
  i had a similar thought when i read this. it would be nice if you can share other similar stuff in that list of yours! :-)
jwise0 12y ago
In a similar family, note also that setuid() can fail! If you try to setuid() to a user that has has reached their ulimit for number of processes, then setuid() will fail, just like fork() would for that user. This is a classic way to get your application exploited. Google did it (at least) twice in Android: once in ADB [1], and once in Zygote [2]. Both resulted in escalation. Check your return values! All of them! [1] http://thesnkchrmr.wordpress.com/2011/03/24/rageagainstthecage/ http://thesnkchrmr.wordpress.com/2011/03/24/rageagainsttheca... [2] https://github.com/unrevoked/zysploit https://github.com/unrevoked/zysploit
- agwa 12y ago
  Thankfully, setuid() no longer fails on Linux because of RLIMIT_NPROC, as of 2011[1]. Still, I agree with you 100%: check your syscall return values, especially security-critical syscalls like setuid! [1] http://lwn.net/Articles/451985/ http://lwn.net/Articles/451985/ and http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=72fa59970f8698023045ab0713d66f3f4f96945c http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.g...
kazinator 12y ago
I once did rm -rf $PREFIX/usr/lib in a Bash script being run as root. PREFIX was misspelled, and set -u was not in effect, so the misspelled variable silently expanded to nothing ...
- Macha 12y ago
  Or Bumblebee's pretty infamous by now: rm -rf /usr /lib/bumblebee
quotemstr 12y ago
I wish posix_spawn were ubiquitous; it's a much better process-launching interface than fork: it's naturally race-free and amenable to use in multi-threaded programs, and unlike fork(2), it plays well with turning VM overcommit off. (If overcommit is off and a large process forks, the system must assume that every COW page could be made process-private and reserve that much memory. Ouch.)
- cperciva 12y ago
  Even if VM overcommit is enabled, you shouldn't be forking from large processes, because the necessary kernel VM manipulation will kill your performance.
  - quotemstr 12y ago
    Yep. For that problem, there's vfork.
    - cperciva 12y ago
      Which is great, albeit easy to use wrong; but it's not portable, unfortunately.
      - quotemstr 12y ago
        It's portable enough: Linux, Darwin, and the BSDs all support it. Darwin also supports posix_spawn natively. Even Cygwin supports it, although Cygwin's vfork is currently just an alias for fork.
        girvo 12y ago
        I seem to remember Cygwin having all sorts of weird problems with fork behaviour, but I'm no C programmer let alone doing stuff on Windows, so I might be remembering incorrectly
- FooBarWidget 12y ago
ionelm 12y ago
Seems Python handles this correctly (by raising an exception): >>> resource.setrlimit(resource.RLIMIT_NPROC, (0, 0)) >>> os.fork() Traceback (most recent call last): File "<ipython-input-7-348c6e46312a>", line 1, in <module> os.fork() OSError: [Errno 11] Resource temporarily unavailable
- illumen 12y ago
  There are plenty of little nice things like this in python that save you. So if you're porting python code to other platforms... be very careful!
IgorPartola 12y ago
Back in the day I had a Motorola Atrix (remember those? First dual core Android phone, best thing since sliced bread, abandoned by Motorola a few months after launch?). Well, one of the ways to root it was to keep forking a process until the phone ran out of memory. After fork failed, you were left with a process that for some reason was running with root privileges...
- TheLoneWolfling 12y ago
  Probably tried to setuid and didn't bother to check the result.
CSDude 12y ago
I'm a teaching assistant of an OS course, I grade projects. I constantly remind students to check the return values of the system calls and it is mostly the main issue in their codes.
brazzy 12y ago
And this right there is why exceptions are a superior mechanism of announcing errors...
- andrewchambers 12y ago
  Multiple return would be fine too. pid,err = fork()
  - knocte 12y ago
    Are you sure? And what if $programmer forgets to check what's in err? What would pid contain in that case? I mention this because I guess you quoted a kind of syntax that matches the one from Go. So then I'm guessing that Go would simply ignore the error in this case. However, having a proper exception mechanism, if you don't catch the problem, then it bubbles up, and the program doesn't continue with wrong data (which is a good thing => fail fast!). This is the biggest downside of GoLang IMNSHO.
    - alxnlssn 12y ago
      The compile will fail if err is unused. You have to explicitly ignore it by doing like this: pid,_ = fork()
      - donatj 12y ago
        And these underscores are really easy to find. Then you just do a git blame and get your shaming on at code review time.
    - donatj 12y ago
      Actually if you want to not handle an error, you have to do either _ = err or just go data, _ = doStuff(), both of which are very visible. You can basically scan your codebase for _'s and find all the unhanded exceptions. If you don't do something with a variable, eg I do myInt := 1 but I don't use myInt go simply refuses to compile.
yokom 12y ago
I don't use fork() that often, but my own paranoia is why I always test for <= 0 instead of == 0. Some people think I'm weird for doing something like: if len(some_list) <= 0: # Test for empty list But it's just my way of covering my ass in case the laws of physics change during execution, or just in case weird bugs exist like those found in this article.
- kelnos 12y ago
  Perhaps I'm misunderstanding what you're saying, but that's wrong too. With fork() you need to always test 3 cases: * -1: error * 0: success, in child * > 0: success, in parent Testing for <= 0 would cause you to think there's an error when you're really just the child process.
  - dllthomas 12y ago
    Or that you're the child process when really there's an error, which might go undetected longer.
    - djcapelis 12y ago
      Thankfully that case will never happen because you can't be a child if there was an error. :) Also the check as presumably written would never miss an error, it would just potentially assume valid return values were also errors.
      - dllthomas 12y ago
        "Thankfully that case will never happen because you can't be a child if there was an error. :)" Uh, no. If there is an error, there will be no child process but the parent process will think it is the child. From the fork man page, emphasis mine: "On success, the PID of the child process is returned in the parent, and 0 is returned in the child." "Also the check as presumably written would never miss an error, it would just potentially assume valid return values were also errors." That was already discussed as a possibility; I was addressing the other. In the case you describe the software would never work at all, even when fork successfully forks, because the child will always think there was an error and presumably fall over rather than getting things done. That's probably the better case, in terms of development progress, because it would be spotted and fixed right away. But hopefully fixed correctly, and not converted to the broken-but-working-when-fork-succeeds other variant that also uses "<= 0".
ajarmst 12y ago
Stevens and Rago, "Advanced Programming in the Unix Environment, Volume II", page 211,212. if ((pid = fork()) < 0) err_sys("fork error"); is idiomatic in Unix.
- wiml 12y ago
  err_sys() is something you have to provide yourself, though, which takes you out of the flow of the code you're thinking about, which is half the reason people don't write error handling in the first place. Once I discovered the existence of the BSD err()/errx()/warn() functions, though, the error handling in even my quick one-off programs became much better and more informative. pid_t child = fork(); if (child < 0) { err("fork"); } else if (child == 0) { ... in child } else { ... in parent } is idiomatic, quick to write, and produces useful error messages when that "throwaway" program starts failing years later.
AnimalMuppet 12y ago
Somewhat OT, but in the same neighborhood: Standard file handles are another thing you should not assume are there (though I'm not sure how to test for it programmatically). We once had a user that, for whatever reason, tweaked their Unix installations to not pass an open stderr to processes - they just got stdin and stdout (that is, file handles 0 and 1, but not 2). If you wrote to stderr anywhere in your program, it wrote to whatever was open on handle 2, which was not a stderr that the OS passed in. Yeah, that's a pretty insane thing to do, but somebody was doing it...
- djcapelis 12y ago
  Wow that's a really wild story, but I think it's also pretty different. fork() returning -1 is defined behavior, as is true for many functions. Whereas not having stderr open defies everything about the C standard I/O. (K&R B1, 7.5 & 7.6) Note however, that strictly speaking stderr does not have to be 2. It can be any number, but it has to be whatever the include file says it is, so if you don't specify the stream as stderr but instead write to stream 2, that would potentially be a problem. That said, if for some reason you have a program completely defying the C standard, you can test whether the streams are open (and they are explicitly defined as having to be open) using fcntl and testing for EBADF at the very beginning of the program.
  - __david__ 12y ago
    > Note however, that strictly speaking stderr does not have to be 2. It Nope. On POSIX systems stderr is defined as 2: http://pubs.opengroup.org/onlinepubs/9699919799/functions/stdin.html http://pubs.opengroup.org/onlinepubs/9699919799/functions/st... > The following symbolic values in <unistd.h> define the file descriptors > that shall be associated with the C-language stdin, stdout, and stderr > when the application is started: > > STDIN_FILENO > Standard input value, stdin. Its value is 0. > STDOUT_FILENO > Standard output value, stdout. Its value is 1. > STDERR_FILENO > Standard error value, stderr. Its value is 2.
    - djcapelis 12y ago
general_failure 12y ago
This is why we need checked exceptions. Imagine if C/POSIX had a checked ChildProcessCloneException
smegel 12y ago
I see it happen quite often on boxes with limited memory and hungry processes.
JD557 12y ago
Is there any clean way to use an Option/Maybe monad in C (or C++)? It should be a simple way to solve problems where error codes are valid inputs of other functions. The simplest way I can think of is: struct maybe { bool isEmpty; void* value; } Although I wonder if using C++ templates, classes and operator overloading is possible to make a more practical implementation (using void* does seem like a bad idea).
- sbmassey 12y ago
  In C++ std::optional<T> does the trick, as of version 11, and boost::optional<T> previously. In C ... hrm you could probably wrangle some macros around that struct if you were desperate.
  - pbsd 12y ago
    Nitpick: std::optional<T> does not exist in C++11, nor in the recently-approved C++14. It is, nevertheless, in the process of getting into the standard, and is already present in GCC 4.9's libstdc++ as std::experimental::optional.
  - whyever 12y ago
    Unfortunately `std::optional` did not get accepted for C++14, because it's implementation uses some hacks that are undefined behavior by the standard (but work on the major compilers).
- MichaelGG 12y ago
  The problem is solved, but consider the overhead of all this. You now need an extra byte somewhere (register, stack space, whatever) to track the tag. I would imagine that this, and lack of handy syntax, is why lower-level APIs don't offer better return values.
wmil 12y ago
The kill -1 behaviour seems like a bug. Sure, it's documented, but how often is it done on purpose? It seems like something that should at least be a separate function.
- dllthomas 12y ago
  If I set off something that proves to be a fork bomb as an unprivileged user, kill -1 might be a reasonable approach. And note that kill exists as a builtin in bash, so you can run it without forking a process. Hopefully doesn't happen often, but potentially very useful in narrow circumstances. The problem is it being -1, not it being available. If it took an argument to kill that you'd never accidentally generate as a PID then it might as well be a different function. Of course, with negative numbers otherwise referring to process groups, there's not a lot of room remaining, so yeah...
nikita 12y ago
Yes, fork can fail and we ran into this a few years ago at MemSQL. The problem was that MemSQL would allocate a lot of memory and linux wouldn't allow to fork such a process. A remedy to that is to create a separate process and talk to it via tcp. This small and low on memory consumption process is responsible for fork/exec paradigm.
- leni536 12y ago
  Why doesn't it use vfork() for fork+exec? AFAIK vfork() doesn't clone the allocated memory of the parent process and is only useful for calling exec immediately after forking. http://linux.die.net/man/2/vfork http://linux.die.net/man/2/vfork
donatj 12y ago
And this is why I think Go-lang and its multi-return is the way of the future. In Go you are required to handle errors. If you want them to go away you have to explicitly use an _ and thats really easy to find in the code and shame the person who did it. Nothing fails silently. Nothing fails via primary return. It is such greatness it is hard to express.
- nwmcsween 12y ago
  You can easily do multi-return in c, return a struct it just doesn't hold your hand.
  - donatj 12y ago
    But as a language construct and the agreed upon way of returning errors it's much more powerful. Returning a structure you can just ignore the error piece. Returning an error however in Go, you have to explicitly handle it. If you don't, it's literally a compile error.
- jbb555 12y ago
  Or exceptions in many languages for error conditions that you don't want to accidentally ignore.
- masklinn 12y ago
  > And this is why I think Go-lang and its multi-return is the way of the future. Using multiple returns for such situations is terrible, especially doing so in the way Go does.
azinman2 12y ago
I see a lot of comments blaming the programmer. This is completely the wrong attitude. Why are you treating the programmer like a machine? They're not a machine -- they're human. Regardless if they fully understand the API or not things should have have sane defaults for HUMAN FACTORS reasons. Bugs will always exist. The fact that the Linux kernel has many bugs is just one example of a code base that has over a decade of work put into it by many people with high skill shows that bugs are inevitable. The goal should be to assume people will do stupid things and make fatal behavior more explicit/difficult. Do we really need -1 for kill to do such behavior? How common is that anyway? It's a pretty destructive behavior, and probably should be removed from kill. The human factors approach would say if you really want that behavior then write a for loop to do over the list of pids, because it should never be within easy reach especially for such an uncommon scenario. Apple's iOS API is similar. Try to insert a nil object into an array? Crash. Try to reload an item in a list that's past the known objects index? Crash. So instead of doing something sane like reloading the entire list, the user has a shit experience because off by one errors happen easily especially in front-end/model work [1 re: fb's persistent unread chat]. Not recognizing the human part of things leads to issues everywhere.. reminding me of this article on human factors in health care previously posted on HN [2]. Conclusion: design for humans and default to non-fatal situations. [1] http://facebook.github.io/flux/ http://facebook.github.io/flux/ [2] http://www.newstatesman.com/2014/05/how-mistakes-can-save-lives http://www.newstatesman.com/2014/05/how-mistakes-can-save-li...
- aroman 12y ago
  In general I agree with your "don't blame the programmer" point, but I would seriously hesitate to criticize fork(). Yes, in 2014, that behavior seems uncommon and it seems like very poor design to lump such destructive behavior into an otherwise meaningless "-1"... but remember that fork was not written in 2014. It was written forty-five years ago. I'm not saying it was a great API design decision back then, but I'm willing to bet that it seemed a lot less "wrong" at the time.
  - cpncrunch 12y ago
    I think azinman2 was criticizing kill(), not fork(). It would make more sense to have a separate system call, something like killall().
quackerhacker 12y ago
Sometimes these threads are just serendipity! I just recently finished a multithreaded program where I found obtaining the pid [on linux: getpid()] of child processes spawn was only effective by utilizing a common pipe that was non-blocking [fcntl(pipefd[1], F_SETFL, O_NONBLOCK) ]. In other, more humorous words, as a "parent," it's great to know what your "child," is doing (or in this sense), who your child is (the actual pid), instead of just kill SIGTERM them.
serve_yay 12y ago
Out of "-1 as failure return value", and "-1 to signal all possible processes", at least one is a bad idea.
- leni536 12y ago
  It's the second one. kill() shouldn't have the kill(-1,...) functionality, there should be a separate syscall for this.
Mister_Snuggles 12y ago
This reminds me of the time I was telnet'd (since SSH wasn't a thing at the time) into a remote SunOS/Solaris server. At the time my only Unix experience was with Linux. "killall -9 httpd" gave an unhelpful error message. "killall httpd" also gave an unhelpful error message. "killall", which would give you usage instructions in Linux, killed all processes on the system. Reading this article makes me figure that killall was likely a frontend to kill(-1, ...). That day I learned a valuable lesson about reading man pages and understanding that not all unixes are the same.
- tracker1 12y ago
  That's funny... I almost always try "command --help" first if I'm not sure. Of course some may point out "man command" but I always find man painful, and revert to google.
  - acdha 12y ago
    At least one hairy old Unix killall took no arguments – so "killall --help" was just as bad…
  - Mister_Snuggles 12y ago
    I think I might have done that too, but I can't remember. This was quite a few years ago.
danielbhall001 12y ago
Lol this is a true war HAHA..
prasoon2211 12y ago
The first time I learnt of fork (from an OS book), the example had three branches to the if statement after fork - and the first tested for a negative pid. I suspect that the reason this link has 400 odd upvotes is because more people aren't learning OS the correct way in the beginning. Or maybe my OS book was nice. IDK.
- lchengify 12y ago
  This. One of the most useful classes I took in undergrad was implementing a scheduler and an I-node disk. Nothing shows you all the ways fork() can fail like having to implement it. Every system call can fail, even if it doesn't do something obvious like use disk resources. Ignoring this is how subtle bugs appear that seem unreproducible until you implement correct error handling.
trippy_biscuits 12y ago
"Unix: just enough potholes and bear traps to keep an entire valley going." If you don't understand how to use sharp tools, you may hurt yourself and others. Documentation for fork() clearly explains why and when fork() returns -1. Those that find the man page lacking or elusive may get more out of an earnest study of W. Richard Stevens' book, Advanced Programming in the UNIX Environment. In any case, every system programmer should own a copy and understand its contents.
- Oculus 12y ago
  every system programmer should own a copy I'd argue every programmer. It's such a fundamental part of computers & operating systems that key concepts will come up again and again. Just the other day I wanted to learn about Docker/CoreOS/etcd only to realize that I have an embarrassingly lacking understanding of how UNIX works. I immediately went to the library to pick up this book and begin fixing a flaw of mine (even as a web developer).
  - TheEzEzz 12y ago
    Meh. Not all programmers are on a UNIX system. Not all programmers are even on UNIX || Window.XX. But even if there was only UNIX... the entire point of a well designed system is to allow users of the system to reason about it on a high level, not a domain-expert level or even domain-intermediate level. As programmers we reason about code without worrying too much about gate layout on silicon. As non-system programmers we should likewise not need to worry about shoddy OS design.
- ben336 12y ago
  > If you don't understand how to use sharp tools, you may hurt yourself and others. It's still bad API design when naively handling an error case kills everything. Is there an inherent reason that the error value for a pid has to be the same as the "all pids" value? Unless there's a very compelling reason, it seems like very poor design, well documented or not.
  - din-9 12y ago
gre 12y ago
C programming 101.
jdrago999 12y ago
Yes, yes it can. That's why it's always: fork() or die "Cannot fork: $!";
kabdib 12y ago
When I was young and really didn't understand Unix, my friend and were summer students at NBS (now NIST), and one fine afternoon we wondered what would happen if you ran fork() forever. We didn't know, so we wrote the program and ran it. This was on a PDP-11/45 running v6 or v7 Unix. The printing console (some DECWriter 133 something or other) started burping and spewing stuff about fork failing and other bad things, and a minute or two later one of the folks who had 'root' ran into the machine room with a panic-stricken look because the system had mostly just locked up. "What were you DOING?" he asked / yelled. "Uh, recursive forks, to see what would happen." He grumbled. Only a late 70s hacker with a Unix-class beard can grumble like that, the classic Unix paternal geek attitude of "I'm happy you're using this and learning, but I wish you were smarter about things." I think we had to hard-reset the system, and it came back with an inconsistent file system which he had to repair by hand with ncheck and icheck, because this was before the days of fsck and that's what real programmers did with slightly corrupted Unix file systems back then. Uphill both ways, in the snow, on a breakfast of gravel and no documentation. Total downtime, maybe half an hour. We were told nicely not to do that again. I think I was handed one of the illicit copies of Lions Notes a few days later. "Read that," and that's how my introduction to the guts of operating systems began.
- derefr 12y ago
  > ...a minute or two later one of the folks who had 'root' ran into the machine room with a panic-stricken look because the system had mostly just locked up. It's kind of weird that, while root has always had e.g. 5% reserved disk space on the rootfs for emergencies, one thing no Unix has ever done is enforce a 5% CPU reservation for root so administrators can "talk over" a cascading failure. I think this is possible just recently in Linux with CPU namespacing, but it's still not something any OS does by default.
  - mjevans 12y ago
    It would only be possible if a limit were enforced on all non-primary namespaces. However something that has been /possible/ for a while (but not in practice done) would be to elevate root process priority over other processes. Probably not done due to daemons needing to run as root (which is decreasing as they're able to drop privileges these days).
jacquesm 12y ago
That's why you read the manpage on a function before you apply it rather than just cutting-and-pasting the first bit of code google returns when you search for 'fork example unix'. (In this particular case that actually returns (for me) a bit of code that gets it right.)
jbb555 12y ago
This to me is a good example of why exceptions in modern languages are good way to handle errors. In this case the user has basically ignored the error return from fork() and the accidentally used it in kill. If fork() had thrown an exception for an unexpected failure then the user could not have accidentally ignored it in the same way. I realize that this is not appropriate for a system call but it seems like a good example of why handling errors using exceptions is helpful sometimes.
jheriko 12y ago
i've never actually used fork... feeling glad now. i probably would have not realised this... reminds me of allocating memory for an error message to tell someone they are out of memory. :)
alan-crowe 12y ago
Is there a test command that asks the operating system to run a program but cause the nth fork to fail? I would be more diligent about writing code that handles rare errors if I could create test cases. Writing code that I cannot test feels wrong.
mcguire 12y ago
If you have set a non-root user's process limits correctly, sending SIGKILL to all of that user's processes is likely a perfectly fine response to their fork() failing. If you haven't limited the number of processes a given non-root user can start to some value the machine can handle, sending SIGKILL to all of the user's processes is probably not going to do anymore damage. If a program running as root doesn't correctly handle fork() failing, someone needs to be taken out back and beaten with a stick. Maybe the person who wrote the program, maybe the person who ran it as root. But somebody.

760 points by dantiberian 12y ago

Quietly goes to check the last piece of C I wrote containing a fork(): if (daemon && !test_mode) { int pid = fork(); if (pid == -1) { fatal_error("Failed to fork"); } if (pid != 0) { write_pid(pid_file, pid, !test_mode); exit(0); } } else { write_pid(pid_file, getpid(), !test_mode); } Phew!

cperciva 12y ago
I usually use switch with fork: if (daemon && !test_mode) { int pid; switch (pid = fork()) { case -1: /* Error */ fatal_error("Failed to fork"); case 0: /* In child */ break; default: /* In parent */ write_pid(pid_file, pid, !test_mode); exit(0); } } else { write_pid(pid_file, getpid(), !test_mode); }
- jgrahamc 12y ago
  Yes. That's rather nice. Although I'm not a fan of the (pid = fork()) inline assignment and condition. But that's a matter of taste, not technology.
  - cperciva 12y ago
    Sure, you could have pid = fork(); switch (pid) if you prefer. I find that the inline assignment-and-condition style is clearer since it reads to me as "switch on the result of fork, and cache that value somewhere" whereas the separate statements read to me as "call fork" and "switch on the process ID".
- colanderman 12y ago
  For daemonization, daemon(3) is better (EDIT: assuming you only care about Linux). (It also chdirs to /, closes STD*, and detaches from the terminal.)
  - cperciva 12y ago
    Alas, daemon(3) is not POSIX.
  - mindajar 12y ago
    daemon(3) isn't part of POSIX, and the OS X man page says "the use of this API is discouraged in favor of using launchd(8)", so who knows what might happen here in the future. If you don't care about portability, it's an easier call to make.
- cliveowen 12y ago
  You should always put a break statement at the end of a default branch. The default case is put at the end as a convention but no one (certainly not the C standard) prevents you from adding a new case after, which will be promptly executed even though probably it's what you want. I think this is mentioned a best practice in the K&R.
  - cperciva 12y ago
    You mean, after exit(0) returns?
    - AnimalMuppet 12y ago
      Not after exit(0) returns, no. Instead, you're going to want it after some programmer removes exit(0) six months from now.
      - loup-vaillant 12y ago
        I'm going to write a pre-processor which adds the appropriate `break` statements automatically. No more fall-through bug. (And if you really want to fall through, I could add a `fallthrough` keyword.)
        cperciva 12y ago
        For what it's worth, it's a style bug in FreeBSD to have a fallthrough which does not have a /* FALLTHROUGH */ comment.
        scintill76 12y ago
        Yep, could even re-use the "continue" keyword.
      - hamburglar 12y ago
        smart-ass comment: you should put two breaks in for when some programmer removes one of them six months from now. ;) edit: i know it's not totally analogous since removing the exit and not noticing there's no break is a lot more likely than just randomly removing a break, but the "let's prevent someone clumsy from screwing this code up in the future" argument always makes me laugh a little.
- rosser 12y ago
  I'm kinda surprised that the assignment-during-test thing doesn't generate a warning in the case of a switch statement. It does in other cases (if and while do under gcc and clang, at least).
  - cperciva 12y ago
    If it creates warnings, you can silence them by adding extra parentheses, e.g., if ((buf = malloc(buflen)) == NULL) goto outofmemory;
    - rosser 12y ago
      I was originally going to comment saying exactly that — under gcc and clang, at least, just the extra parens will disable the warning; no comparison necessary — but thought to test that the warning is generated in case of a switch statement (and is then disabled by the extra set of parens), which led to realizing the behavior is different. Just surprised by the inconsistency.
    - loup-vaillant 12y ago
      Careful with that example. The parentheses here don't just remove the warning. They remove a bug. This code: if (buf = malloc(buflen) == NULL) goto outofmemory; is actually equivalent to that code: if (buf = (malloc(buflen) == NULL)) goto outofmemory; So, malloc gives you a pointer, which is compared to the null pointer, giving you either 0 or 1. And that is assigned to buf. Hopefully your compiler will warn you about the type error that spawns from such dark magic.
      - deleted 12y ago
        [deleted]
      - cperciva 12y ago
        Oops, quite right. I was originally thinking of if ((rc = pthread_mutex_lock(mtx))) err("mutex_lock failed: %s", strerror(rc)); but decided to switch to a better-known function at the last minute and completely lost the point.
- ChuckMcM 12y ago
  Somewhat pedantically, I discovered a bug where a daemon (which did this very similar code) started killing random processes. The issue turned out to be that it was test run as root once and the pid_file was owned by root, group root. So the write_pid() function failed (silently) and the cleanup script always took the pid that was in pid_file and sent it a kill 9, which was now stuck. Sometimes kill 9 would return invalid pid, sometimes it would kill some process. Fixed by removing the pid_file, letting the daemon create it, and checking that the write succeeded.
jervisfm 12y ago
I was curious too about my own code, so I looked at the last publicly available C code I wrote with fork in it[1], and yep I checked for the error case too :) I think this just came from being drilled in school on the importance of checking error return values. No matter how unlikely never assume that something can't happen. If it really can't you should at the very least assert on it. [1] - https://github.com/jervisfm/W4118_HW1/blob/master/shell/shell.c#L300 https://github.com/jervisfm/W4118_HW1/blob/master/shell/shel...

cperciva 12y ago

This reminds me of one of the most epic bugs I've ever run into: mkdir("/foo", 0700); chdir("/foo"); recursively_delete_everything_in_current_directory(); Running as root, this usually worked fine: It would create a directory, move into it, and clean out any garbage left behind by a previous run before doing anything new. Running as non-root, the mkdir failed, the chdir failed, and it started eating my home directory.

stinos 12y ago
When you see chdir, or any notion of the current working directory being used for anything: run as fast as you can. (or refactor if it's not too late). Things I've seen because of software relying on it.. Sometimes it's just directories/files it creates popping up all over the place, sometimes it's 'just' crashing, but yes sometimes it starts to erase and all hell really breaks loose.
- cperciva 12y ago
  In this particular case, I didn't see the chdir until after the fact. This wasn't my code...
- dllthomas 12y ago
  Right, better to use the *at() functions (e.g. http://linux.die.net/man/2/unlinkat http://linux.die.net/man/2/unlinkat) where applicable.
- agwa 12y ago
  Sometimes there are legitimate reasons for relying on the CWD, such as avoiding time-of-check/time-of-use race conditions. The *at syscalls provide a better alternative, but (at least as of a few years ago) they weren't widely implemented outside of Linux and Solaris.
- pachydermic 12y ago
  If you can't rely on current working directories then you have to specify any file locations absolutely? That doesn't seem like a good idea because then your code quickly turns into a hot mess if you ever have to change where stuff lives. This is such a stupid problem I run into a lot. Both alternatives (doing things with absolute paths vs doing things entirely with relative paths) seem to have a lot of downsides. Overall relative paths seems to be way better, but then you leave yourself open to problems which "rhyme" with the one OP was talking about.

runarb 12y ago

Easy to test out to. In C: #include <unistd.h> int main(void) { while(1) { fork(); } }

tdicola 12y ago
Haha too cruel. I remember in one of my early college CS classes the professor told us about fork bombs and wasn't sure if they still worked on the Sun Fire servers we were using. About 5 minutes later someone piped up and confirmed that yep they still worked at bringing the server down. :)
- X-Istence 12y ago
  That's why user accounts should have appropriate resource limits set per user.
arthurcolle 12y ago
while(fork()) {};
- TheLoneWolfling 12y ago
  while(1) {fork()}; is "better", as the version you mention will quit as soon as it is able to fork a new process.
  - jimm 12y ago
    Will it quit? A failed fork() will return -1, which is non-zero thus true in a boolean context. The while(fork()) {...} will run.
    - comex 12y ago
      But it will stop when fork returns 0, i.e. in the child processes, making it non-exponential.
  - 12y ago

Aurel1us 12y ago

Just as a reminder: "So, malloc on Linux only fails if there isn’t enough memory for its control structures. It does not fail if there isn’t enough memory to fulfill the request." - http://scvalex.net/posts/6/ http://scvalex.net/posts/6/

stonogo 12y ago
This is a mostly-untrue statement because it makes unreliable assumptions about the host system. It depends on the vm.overcommit_memory setting and the programmer should never make assumptions about why or when malloc might fail. Read more on Rich Felker's excellent blog post here: http://ewontfix.com/3/ http://ewontfix.com/3/
acdha 12y ago
That page is incomplete: malloc() will fail on Linux depending on the `vm_overcommit` setting: https://www.kernel.org/doc/Documentation/vm/overcommit-accounting https://www.kernel.org/doc/Documentation/vm/overcommit-accou...
quotemstr 12y ago
Malloc can also fail if you're out of address space without necessarily being out of memory. NT does a much better job of separating these concepts than Unix-family operating systems do. Conceptually, setting aside a region of your process's address space and guaranteeing that the OS will be able to serve you a given number of pages are completely different operations. I wish more programs would use MAP_NORESERVE when they want the former without the latter. (I'm looking at you, Java.) One day, perhaps when I am old and frail, we will achieve sanity and turn overcommit off by default. But we're a long way from being able to do that now.
- klodolph 12y ago
  These days, I describe malloc() as "a function which allocates address space", to avoid confusion. Which means it makes sense that malloc() returns NULL if you are out of address space, even if you have lots of memory. (But so many people don't check malloc()'s return anyway...)

zokier 12y ago

Who needs type safety when we got integers.

zxcdw 12y ago
Who even wants that? When I am coding, sure as hell some compiler vendor won't be telling me what to use my bits for! I identify myself as an atypical coder.
- kazinator 12y ago
  If you tell yourself what the bits are for and stick to it, that is still type. Real chaos is when even you don't.
  - deleted 12y ago
    [deleted]
AnimalMuppet 12y ago
Is this just using some vague connection to ride on one of your favorite hobby horses? Or does this have a connection to the article, and I missed it?
- GhotiFish 12y ago
  well you're no fun.
- mkehrt 12y ago
  The problem is that C and POSXIX don't (idiomatically) provide rich enough data types force checking the error condition. The fact that the error is signaled by a random integer (-1) is horrible. In a language with stronger types and richer data structures, one can have a return type that is a disjunction of {failure, parent, child}, so you can never accidentally treat a failure as a PID. In a functional language this would look like a datatype (essentially a generalized enum), while an OO language you would use different subclasses of a common superclass. In C you can return a tagged union and check the tag, but nothing forces you to do the check. A user of this API can just go ahead and assume the success branch of the union. Furthermore, this isn't idiomatic POSIX, so it is never done. [edit: "subclass" -> "superclass"]

emo_tards_on_hn 12y ago

This is an important point, along similar lines to something we discussed at work today in a code review. The programmer in question assumed that all and every request for the resources consumed would always work. Literally, no testing or consideration for what if one of those request fails. Will be emailing them this article tomorrow!

arbitrage 12y ago
Your username is offensive. I'd like to take what you say seriously, and perhaps even engage in further conversation about the topic at hand, but ... you know ... you look stupid. Grow up. You should be embarrassed.
- dang 12y ago
  That's understandable, but please don't feed the trolls. When you see a comment that is truly bad for HN, it's best to flag it. You can do this by clicking on "link" to go to the item page for the comment, then "flag". We monitor those flags and take action based on them.
- stinos 12y ago
  Shouldn't take long user: emo_tards_on_hn created: 48 minutes ago karma: -20
- slingerofwheat 12y ago
  So. You mean you actually DON'T like to take what he said seriously. His username is offensive, but you just also gave it credibility.
  - schrodinger 12y ago
    "I'd like to" means I would like to, presumably meaning "I would like to take your post seriously, but your offensive handle is preventing me"

japaget 12y ago

See also https://news.ycombinator.com/item?id=8189968 https://news.ycombinator.com/item?id=8189968 for another UNIX trap for the unwary.

mnw21cam 12y ago
I sometimes drop a "--help" or "--version" file in a directory I don't want to accidentally run "rm *" in.

dasmithii 12y ago

I'm uncertain that I've ever checked for errors after calling fork. Thanks for providing an impetus to do so.

mutation 12y ago

Just noticed that in Perl the behavior is slightly different: http://perldoc.perl.org/functions/fork.html http://perldoc.perl.org/functions/fork.html unsuccessful fork() returns undef, effectively stopping you from kill-ing what you don't want to kill.

epochwolf 12y ago
In C, there is no concept of null or undefined, only 0.
- dllthomas 12y ago
  That's not quite true, but not a terrible approximation.
  - sdegutis 12y ago
    How is it not true? Besides void expressions (which aren't values) the above poster's statement appears correct.
    - dllthomas 12y ago
      A null pointer does not need to have an all-zero representation, which mattered once on some architectures. I'm not aware of any current architecture that does it that way, but that's still what the standard says, which is to say that C does technically have a notion of null which is not necessarily identical with 0. http://c-faq.com/null/machexamp.html http://c-faq.com/null/machexamp.html
      - spc476 12y ago
        But in C, a 0 in a pointer context is always considered the NULL pointer (even if the physical value of a NULL pointer isn't all zeros).
        loup-vaillant 12y ago

spudlyo 12y ago

If a function be advertised to return an error code in the event of difficulties, thou shalt check for that code, yea, even though the checks triple the size of thy code and produce aches in thy typing fingers, for if thou thinkest "it cannot happen to me", the gods shall surely punish thee for thy arrogance. [0] [0]: http://www.lysator.liu.se/c/ten-commandments.html http://www.lysator.liu.se/c/ten-commandments.html

leni536 12y ago
Note that wasn't the real reason. At the bottom you can see an edit which reads: "I was wrong about why malloc finally failed! @GodmarBack observes, in the comments, that x64 systems only have an address space of 48 bits, which comes out to about 131000 GB. So, on my machine at least, the malloc finally failed because of address space exhaustion."
- eshyong 12y ago
  FYI, I think you replied to the wrong message :)
quotemstr 12y ago
Counterexample: pthread_mutex_unlock. That function returns an error code, but it cannot possibly fail in a well-formed program. Checking for an error for mutex unlock is pointless: what would you do in response?
- cpeterso 12y ago
  > cannot possibly fail in a well-formed program I think that's your answer. A mutex error return probably indicates an application bug, such as double unlock. You should probably assert or abort on "can't happen" mutex errors. Programmers are lazy. If they take the time to document an error return value, then you should probably heed their warnings. :)
  - quotemstr 12y ago

VLM 12y ago

"Neither of them fail often" See /etc/security/limits.conf and nproc and "fork bomb" Aside from intentional fork bombs I've seen this done intentionally in the spirit of a OOMkiller to keep a machine alive for debugging / detection of problem. 100 "whatever" processes will kill this webserver making it impossible to log in and diagnose much less fix, so we'll limit to 50 processes in the OS. I've also seen it in systems where people are too lazy to test if a process is running before forking another and the system doesn't like multiple copies running (like a keep alive restarter pattern). If ops has no access to the source to fix that or no one cares, then just run it in jail where you only get two processes, the restarter-forker and the forkee. Then hilarity can result if the restarter thinks the PID of the failed fork means something, like sending an email alert or logging the restart attempt. "Why are my logs now gigabytes of ERROR: restarted process new pid is -1?"

walski 12y ago

Thanks! Definitively in my "shit I should know but didn't before HN schooled me"-top-10 :)

vigneshv_psg 12y ago
i had a similar thought when i read this. it would be nice if you can share other similar stuff in that list of yours! :-)

jwise0 12y ago

In a similar family, note also that setuid() can fail! If you try to setuid() to a user that has has reached their ulimit for number of processes, then setuid() will fail, just like fork() would for that user. This is a classic way to get your application exploited. Google did it (at least) twice in Android: once in ADB [1], and once in Zygote [2]. Both resulted in escalation. Check your return values! All of them! [1] http://thesnkchrmr.wordpress.com/2011/03/24/rageagainstthecage/ http://thesnkchrmr.wordpress.com/2011/03/24/rageagainsttheca... [2] https://github.com/unrevoked/zysploit https://github.com/unrevoked/zysploit

agwa 12y ago
Thankfully, setuid() no longer fails on Linux because of RLIMIT_NPROC, as of 2011[1]. Still, I agree with you 100%: check your syscall return values, especially security-critical syscalls like setuid! [1] http://lwn.net/Articles/451985/ http://lwn.net/Articles/451985/ and http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=72fa59970f8698023045ab0713d66f3f4f96945c http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.g...

kazinator 12y ago

I once did rm -rf $PREFIX/usr/lib in a Bash script being run as root. PREFIX was misspelled, and set -u was not in effect, so the misspelled variable silently expanded to nothing ...

Macha 12y ago
Or Bumblebee's pretty infamous by now: rm -rf /usr /lib/bumblebee

quotemstr 12y ago

I wish posix_spawn were ubiquitous; it's a much better process-launching interface than fork: it's naturally race-free and amenable to use in multi-threaded programs, and unlike fork(2), it plays well with turning VM overcommit off. (If overcommit is off and a large process forks, the system must assume that every COW page could be made process-private and reserve that much memory. Ouch.)

cperciva 12y ago
Even if VM overcommit is enabled, you shouldn't be forking from large processes, because the necessary kernel VM manipulation will kill your performance.
- quotemstr 12y ago
  Yep. For that problem, there's vfork.
  - cperciva 12y ago
    Which is great, albeit easy to use wrong; but it's not portable, unfortunately.
    - quotemstr 12y ago
      It's portable enough: Linux, Darwin, and the BSDs all support it. Darwin also supports posix_spawn natively. Even Cygwin supports it, although Cygwin's vfork is currently just an alias for fork.
      - girvo 12y ago
        I seem to remember Cygwin having all sorts of weird problems with fork behaviour, but I'm no C programmer let alone doing stuff on Windows, so I might be remembering incorrectly
FooBarWidget 12y ago

ionelm 12y ago

Seems Python handles this correctly (by raising an exception): >>> resource.setrlimit(resource.RLIMIT_NPROC, (0, 0)) >>> os.fork() Traceback (most recent call last): File "<ipython-input-7-348c6e46312a>", line 1, in <module> os.fork() OSError: [Errno 11] Resource temporarily unavailable

illumen 12y ago
There are plenty of little nice things like this in python that save you. So if you're porting python code to other platforms... be very careful!

IgorPartola 12y ago

Back in the day I had a Motorola Atrix (remember those? First dual core Android phone, best thing since sliced bread, abandoned by Motorola a few months after launch?). Well, one of the ways to root it was to keep forking a process until the phone ran out of memory. After fork failed, you were left with a process that for some reason was running with root privileges...

TheLoneWolfling 12y ago
Probably tried to setuid and didn't bother to check the result.

CSDude 12y ago

I'm a teaching assistant of an OS course, I grade projects. I constantly remind students to check the return values of the system calls and it is mostly the main issue in their codes.

brazzy 12y ago

And this right there is why exceptions are a superior mechanism of announcing errors...

andrewchambers 12y ago
Multiple return would be fine too. pid,err = fork()
- knocte 12y ago
  Are you sure? And what if $programmer forgets to check what's in err? What would pid contain in that case? I mention this because I guess you quoted a kind of syntax that matches the one from Go. So then I'm guessing that Go would simply ignore the error in this case. However, having a proper exception mechanism, if you don't catch the problem, then it bubbles up, and the program doesn't continue with wrong data (which is a good thing => fail fast!). This is the biggest downside of GoLang IMNSHO.
  - alxnlssn 12y ago
    The compile will fail if err is unused. You have to explicitly ignore it by doing like this: pid,_ = fork()
    - donatj 12y ago
      And these underscores are really easy to find. Then you just do a git blame and get your shaming on at code review time.
  - donatj 12y ago
    Actually if you want to not handle an error, you have to do either _ = err or just go data, _ = doStuff(), both of which are very visible. You can basically scan your codebase for _'s and find all the unhanded exceptions. If you don't do something with a variable, eg I do myInt := 1 but I don't use myInt go simply refuses to compile.

yokom 12y ago

I don't use fork() that often, but my own paranoia is why I always test for <= 0 instead of == 0. Some people think I'm weird for doing something like: if len(some_list) <= 0: # Test for empty list But it's just my way of covering my ass in case the laws of physics change during execution, or just in case weird bugs exist like those found in this article.

kelnos 12y ago
Perhaps I'm misunderstanding what you're saying, but that's wrong too. With fork() you need to always test 3 cases: * -1: error * 0: success, in child * > 0: success, in parent Testing for <= 0 would cause you to think there's an error when you're really just the child process.
- dllthomas 12y ago
  Or that you're the child process when really there's an error, which might go undetected longer.
  - djcapelis 12y ago
    Thankfully that case will never happen because you can't be a child if there was an error. :) Also the check as presumably written would never miss an error, it would just potentially assume valid return values were also errors.
    - dllthomas 12y ago
      "Thankfully that case will never happen because you can't be a child if there was an error. :)" Uh, no. If there is an error, there will be no child process but the parent process will think it is the child. From the fork man page, emphasis mine: "On success, the PID of the child process is returned in the parent, and 0 is returned in the child." "Also the check as presumably written would never miss an error, it would just potentially assume valid return values were also errors." That was already discussed as a possibility; I was addressing the other. In the case you describe the software would never work at all, even when fork successfully forks, because the child will always think there was an error and presumably fall over rather than getting things done. That's probably the better case, in terms of development progress, because it would be spotted and fixed right away. But hopefully fixed correctly, and not converted to the broken-but-working-when-fork-succeeds other variant that also uses "<= 0".

ajarmst 12y ago

Stevens and Rago, "Advanced Programming in the Unix Environment, Volume II", page 211,212. if ((pid = fork()) < 0) err_sys("fork error"); is idiomatic in Unix.

wiml 12y ago
err_sys() is something you have to provide yourself, though, which takes you out of the flow of the code you're thinking about, which is half the reason people don't write error handling in the first place. Once I discovered the existence of the BSD err()/errx()/warn() functions, though, the error handling in even my quick one-off programs became much better and more informative. pid_t child = fork(); if (child < 0) { err("fork"); } else if (child == 0) { ... in child } else { ... in parent } is idiomatic, quick to write, and produces useful error messages when that "throwaway" program starts failing years later.

AnimalMuppet 12y ago

Somewhat OT, but in the same neighborhood: Standard file handles are another thing you should not assume are there (though I'm not sure how to test for it programmatically). We once had a user that, for whatever reason, tweaked their Unix installations to not pass an open stderr to processes - they just got stdin and stdout (that is, file handles 0 and 1, but not 2). If you wrote to stderr anywhere in your program, it wrote to whatever was open on handle 2, which was not a stderr that the OS passed in. Yeah, that's a pretty insane thing to do, but somebody was doing it...

djcapelis 12y ago
Wow that's a really wild story, but I think it's also pretty different. fork() returning -1 is defined behavior, as is true for many functions. Whereas not having stderr open defies everything about the C standard I/O. (K&R B1, 7.5 & 7.6) Note however, that strictly speaking stderr does not have to be 2. It can be any number, but it has to be whatever the include file says it is, so if you don't specify the stream as stderr but instead write to stream 2, that would potentially be a problem. That said, if for some reason you have a program completely defying the C standard, you can test whether the streams are open (and they are explicitly defined as having to be open) using fcntl and testing for EBADF at the very beginning of the program.
- __david__ 12y ago
  > Note however, that strictly speaking stderr does not have to be 2. It Nope. On POSIX systems stderr is defined as 2: http://pubs.opengroup.org/onlinepubs/9699919799/functions/stdin.html http://pubs.opengroup.org/onlinepubs/9699919799/functions/st... > The following symbolic values in <unistd.h> define the file descriptors > that shall be associated with the C-language stdin, stdout, and stderr > when the application is started: > > STDIN_FILENO > Standard input value, stdin. Its value is 0. > STDOUT_FILENO > Standard output value, stdout. Its value is 1. > STDERR_FILENO > Standard error value, stderr. Its value is 2.
  - djcapelis 12y ago

general_failure 12y ago

This is why we need checked exceptions. Imagine if C/POSIX had a checked ChildProcessCloneException

smegel 12y ago

I see it happen quite often on boxes with limited memory and hungry processes.

JD557 12y ago

Is there any clean way to use an Option/Maybe monad in C (or C++)? It should be a simple way to solve problems where error codes are valid inputs of other functions. The simplest way I can think of is: struct maybe { bool isEmpty; void* value; } Although I wonder if using C++ templates, classes and operator overloading is possible to make a more practical implementation (using void* does seem like a bad idea).

sbmassey 12y ago
In C++ std::optional<T> does the trick, as of version 11, and boost::optional<T> previously. In C ... hrm you could probably wrangle some macros around that struct if you were desperate.
- pbsd 12y ago
  Nitpick: std::optional<T> does not exist in C++11, nor in the recently-approved C++14. It is, nevertheless, in the process of getting into the standard, and is already present in GCC 4.9's libstdc++ as std::experimental::optional.
- whyever 12y ago
  Unfortunately `std::optional` did not get accepted for C++14, because it's implementation uses some hacks that are undefined behavior by the standard (but work on the major compilers).
MichaelGG 12y ago
The problem is solved, but consider the overhead of all this. You now need an extra byte somewhere (register, stack space, whatever) to track the tag. I would imagine that this, and lack of handy syntax, is why lower-level APIs don't offer better return values.

wmil 12y ago

The kill -1 behaviour seems like a bug. Sure, it's documented, but how often is it done on purpose? It seems like something that should at least be a separate function.

dllthomas 12y ago
If I set off something that proves to be a fork bomb as an unprivileged user, kill -1 might be a reasonable approach. And note that kill exists as a builtin in bash, so you can run it without forking a process. Hopefully doesn't happen often, but potentially very useful in narrow circumstances. The problem is it being -1, not it being available. If it took an argument to kill that you'd never accidentally generate as a PID then it might as well be a different function. Of course, with negative numbers otherwise referring to process groups, there's not a lot of room remaining, so yeah...

nikita 12y ago

Yes, fork can fail and we ran into this a few years ago at MemSQL. The problem was that MemSQL would allocate a lot of memory and linux wouldn't allow to fork such a process. A remedy to that is to create a separate process and talk to it via tcp. This small and low on memory consumption process is responsible for fork/exec paradigm.

leni536 12y ago
Why doesn't it use vfork() for fork+exec? AFAIK vfork() doesn't clone the allocated memory of the parent process and is only useful for calling exec immediately after forking. http://linux.die.net/man/2/vfork http://linux.die.net/man/2/vfork

donatj 12y ago

And this is why I think Go-lang and its multi-return is the way of the future. In Go you are required to handle errors. If you want them to go away you have to explicitly use an _ and thats really easy to find in the code and shame the person who did it. Nothing fails silently. Nothing fails via primary return. It is such greatness it is hard to express.

nwmcsween 12y ago
You can easily do multi-return in c, return a struct it just doesn't hold your hand.
- donatj 12y ago
  But as a language construct and the agreed upon way of returning errors it's much more powerful. Returning a structure you can just ignore the error piece. Returning an error however in Go, you have to explicitly handle it. If you don't, it's literally a compile error.
jbb555 12y ago
Or exceptions in many languages for error conditions that you don't want to accidentally ignore.
masklinn 12y ago
> And this is why I think Go-lang and its multi-return is the way of the future. Using multiple returns for such situations is terrible, especially doing so in the way Go does.

azinman2 12y ago

I see a lot of comments blaming the programmer. This is completely the wrong attitude. Why are you treating the programmer like a machine? They're not a machine -- they're human. Regardless if they fully understand the API or not things should have have sane defaults for HUMAN FACTORS reasons. Bugs will always exist. The fact that the Linux kernel has many bugs is just one example of a code base that has over a decade of work put into it by many people with high skill shows that bugs are inevitable. The goal should be to assume people will do stupid things and make fatal behavior more explicit/difficult. Do we really need -1 for kill to do such behavior? How common is that anyway? It's a pretty destructive behavior, and probably should be removed from kill. The human factors approach would say if you really want that behavior then write a for loop to do over the list of pids, because it should never be within easy reach especially for such an uncommon scenario. Apple's iOS API is similar. Try to insert a nil object into an array? Crash. Try to reload an item in a list that's past the known objects index? Crash. So instead of doing something sane like reloading the entire list, the user has a shit experience because off by one errors happen easily especially in front-end/model work [1 re: fb's persistent unread chat]. Not recognizing the human part of things leads to issues everywhere.. reminding me of this article on human factors in health care previously posted on HN [2]. Conclusion: design for humans and default to non-fatal situations. [1] http://facebook.github.io/flux/ http://facebook.github.io/flux/ [2] http://www.newstatesman.com/2014/05/how-mistakes-can-save-lives http://www.newstatesman.com/2014/05/how-mistakes-can-save-li...

aroman 12y ago
In general I agree with your "don't blame the programmer" point, but I would seriously hesitate to criticize fork(). Yes, in 2014, that behavior seems uncommon and it seems like very poor design to lump such destructive behavior into an otherwise meaningless "-1"... but remember that fork was not written in 2014. It was written forty-five years ago. I'm not saying it was a great API design decision back then, but I'm willing to bet that it seemed a lot less "wrong" at the time.
- cpncrunch 12y ago
  I think azinman2 was criticizing kill(), not fork(). It would make more sense to have a separate system call, something like killall().

quackerhacker 12y ago

Sometimes these threads are just serendipity! I just recently finished a multithreaded program where I found obtaining the pid [on linux: getpid()] of child processes spawn was only effective by utilizing a common pipe that was non-blocking [fcntl(pipefd[1], F_SETFL, O_NONBLOCK) ]. In other, more humorous words, as a "parent," it's great to know what your "child," is doing (or in this sense), who your child is (the actual pid), instead of just kill SIGTERM them.

serve_yay 12y ago

Out of "-1 as failure return value", and "-1 to signal all possible processes", at least one is a bad idea.

leni536 12y ago
It's the second one. kill() shouldn't have the kill(-1,...) functionality, there should be a separate syscall for this.

Mister_Snuggles 12y ago

This reminds me of the time I was telnet'd (since SSH wasn't a thing at the time) into a remote SunOS/Solaris server. At the time my only Unix experience was with Linux. "killall -9 httpd" gave an unhelpful error message. "killall httpd" also gave an unhelpful error message. "killall", which would give you usage instructions in Linux, killed all processes on the system. Reading this article makes me figure that killall was likely a frontend to kill(-1, ...). That day I learned a valuable lesson about reading man pages and understanding that not all unixes are the same.

tracker1 12y ago
That's funny... I almost always try "command --help" first if I'm not sure. Of course some may point out "man command" but I always find man painful, and revert to google.
- acdha 12y ago
  At least one hairy old Unix killall took no arguments – so "killall --help" was just as bad…
- Mister_Snuggles 12y ago
  I think I might have done that too, but I can't remember. This was quite a few years ago.

danielbhall001 12y ago

Lol this is a true war HAHA..

prasoon2211 12y ago

The first time I learnt of fork (from an OS book), the example had three branches to the if statement after fork - and the first tested for a negative pid. I suspect that the reason this link has 400 odd upvotes is because more people aren't learning OS the correct way in the beginning. Or maybe my OS book was nice. IDK.

lchengify 12y ago
This. One of the most useful classes I took in undergrad was implementing a scheduler and an I-node disk. Nothing shows you all the ways fork() can fail like having to implement it. Every system call can fail, even if it doesn't do something obvious like use disk resources. Ignoring this is how subtle bugs appear that seem unreproducible until you implement correct error handling.

trippy_biscuits 12y ago

"Unix: just enough potholes and bear traps to keep an entire valley going." If you don't understand how to use sharp tools, you may hurt yourself and others. Documentation for fork() clearly explains why and when fork() returns -1. Those that find the man page lacking or elusive may get more out of an earnest study of W. Richard Stevens' book, Advanced Programming in the UNIX Environment. In any case, every system programmer should own a copy and understand its contents.

Oculus 12y ago
every system programmer should own a copy I'd argue every programmer. It's such a fundamental part of computers & operating systems that key concepts will come up again and again. Just the other day I wanted to learn about Docker/CoreOS/etcd only to realize that I have an embarrassingly lacking understanding of how UNIX works. I immediately went to the library to pick up this book and begin fixing a flaw of mine (even as a web developer).
- TheEzEzz 12y ago
  Meh. Not all programmers are on a UNIX system. Not all programmers are even on UNIX || Window.XX. But even if there was only UNIX... the entire point of a well designed system is to allow users of the system to reason about it on a high level, not a domain-expert level or even domain-intermediate level. As programmers we reason about code without worrying too much about gate layout on silicon. As non-system programmers we should likewise not need to worry about shoddy OS design.
ben336 12y ago
> If you don't understand how to use sharp tools, you may hurt yourself and others. It's still bad API design when naively handling an error case kills everything. Is there an inherent reason that the error value for a pid has to be the same as the "all pids" value? Unless there's a very compelling reason, it seems like very poor design, well documented or not.
- din-9 12y ago

gre 12y ago

C programming 101.

jdrago999 12y ago

Yes, yes it can. That's why it's always: fork() or die "Cannot fork: $!";

kabdib 12y ago

When I was young and really didn't understand Unix, my friend and were summer students at NBS (now NIST), and one fine afternoon we wondered what would happen if you ran fork() forever. We didn't know, so we wrote the program and ran it. This was on a PDP-11/45 running v6 or v7 Unix. The printing console (some DECWriter 133 something or other) started burping and spewing stuff about fork failing and other bad things, and a minute or two later one of the folks who had 'root' ran into the machine room with a panic-stricken look because the system had mostly just locked up. "What were you DOING?" he asked / yelled. "Uh, recursive forks, to see what would happen." He grumbled. Only a late 70s hacker with a Unix-class beard can grumble like that, the classic Unix paternal geek attitude of "I'm happy you're using this and learning, but I wish you were smarter about things." I think we had to hard-reset the system, and it came back with an inconsistent file system which he had to repair by hand with ncheck and icheck, because this was before the days of fsck and that's what real programmers did with slightly corrupted Unix file systems back then. Uphill both ways, in the snow, on a breakfast of gravel and no documentation. Total downtime, maybe half an hour. We were told nicely not to do that again. I think I was handed one of the illicit copies of Lions Notes a few days later. "Read that," and that's how my introduction to the guts of operating systems began.

derefr 12y ago
> ...a minute or two later one of the folks who had 'root' ran into the machine room with a panic-stricken look because the system had mostly just locked up. It's kind of weird that, while root has always had e.g. 5% reserved disk space on the rootfs for emergencies, one thing no Unix has ever done is enforce a 5% CPU reservation for root so administrators can "talk over" a cascading failure. I think this is possible just recently in Linux with CPU namespacing, but it's still not something any OS does by default.
- mjevans 12y ago
  It would only be possible if a limit were enforced on all non-primary namespaces. However something that has been /possible/ for a while (but not in practice done) would be to elevate root process priority over other processes. Probably not done due to daemons needing to run as root (which is decreasing as they're able to drop privileges these days).

jacquesm 12y ago

That's why you read the manpage on a function before you apply it rather than just cutting-and-pasting the first bit of code google returns when you search for 'fork example unix'. (In this particular case that actually returns (for me) a bit of code that gets it right.)

jbb555 12y ago

This to me is a good example of why exceptions in modern languages are good way to handle errors. In this case the user has basically ignored the error return from fork() and the accidentally used it in kill. If fork() had thrown an exception for an unexpected failure then the user could not have accidentally ignored it in the same way. I realize that this is not appropriate for a system call but it seems like a good example of why handling errors using exceptions is helpful sometimes.

jheriko 12y ago

i've never actually used fork... feeling glad now. i probably would have not realised this... reminds me of allocating memory for an error message to tell someone they are out of memory. :)

alan-crowe 12y ago

Is there a test command that asks the operating system to run a program but cause the nth fork to fail? I would be more diligent about writing code that handles rare errors if I could create test cases. Writing code that I cannot test feels wrong.

mcguire 12y ago

If you have set a non-root user's process limits correctly, sending SIGKILL to all of that user's processes is likely a perfectly fine response to their fork() failing. If you haven't limited the number of processes a given non-root user can start to some value the machine can handle, sending SIGKILL to all of the user's processes is probably not going to do anymore damage. If a program running as root doesn't correctly handle fork() failing, someone needs to be taken out back and beaten with a stick. Maybe the person who wrote the program, maybe the person who ran it as root. But somebody.