4 ms·

> Checking the return value of malloc is a bad idea I am embarassed for you.

• tptacek 12y ago
  That's fascinating. Tell me more.
  • dkhenry 12y ago
    He clearly has never heard of mallopt (http://man7.org/linux/man-pages/man3/mallopt.3.html http://man7.org/linux/man-pages/man3/mallopt.3.html) so he doesn't know that instead of wrapping every single malloc with an if statement you can just tell malloc to abrt if it ever fails to allocate memory. In his defense most programs don't test the return of malloc and don't set the abort on failure so most programmers would just assume that if your not checking your return values that you made a mistake.
  • stonogo 12y ago
    If a daemon is not capable of surviving memory exhaustion, it is not suitable for deployment. If a programmer is not capable of writing software that can survive memory exhaustion, they are not suitable for writing daemons. I'll not argue the point further; the HN lynchmob is already beating at my door for daring to disagree with a site darling. Had I looked at your username before replying, I probably would have forborn from violating the bubble.
    • SamReidHughes 12y ago
      Quoting and with total vacuity, replying, "I am embarassed for you," will get you downvoted no matter who the parent's author is.
    • tptacek 12y ago
      What's a single example of a daemon with more than 10,000 lines of C code that will reliably "survive" memory exhaustion? By "survive" we obviously both mean that it retains its original PID. Of course, most Unix serverside code "survives" by simply aborting and allowing itself to be restarted. Since that's what I'm recommending, that strategy doesn't count. To find one, you're going to have to catalog every malloc() in the entire program and record some kind of recovery regime --- maybe degraded performance, maybe dropped requests, maybe fallback to some kind of pool --- for every allocation.
      • halayli 12y ago
        An in-memory message queue server or memcached variations don't need to crash if it can't serve a particular request that requires more memory than available. I am not sure why error-handling is a difficult concept for you.
        • tptacek 12y ago
          You just moved the goalposts. Yes, it's suboptimal to have a server that receives a request with a 32 bit length field indicating 3 gigs of incoming data that the bombs out trying to malloc 32 gigs. I'm saying: what's an example of a server that reliably, for every allocation of every piece of metadata, every strdup, every hash table entry, every connection object, &c &c, has a recovery regime so that it doesn't have to fail ever when malloc does? One way you could find such a program would be to compile a candidate and preload a malloc that randomly (1 in every 100 calls per allocation size, for instance) returns NULL. See if the program (a) continues to run and (b) passes some simple unit test suite. My contention is: (a) Those programs will be hard to find, because most "production ready" Unix code does not have that property, and (b) The criteria I'm talking about matters a lot, because (1) memory exhaustion strikes at totally arbitrary points in a program's execution, not just at the points where you're prepared to handle it, and (2) attackers can pinpoint exactly the allocation they want to have fail. As an example of an approach that doesn't seem to work well: in "memqueue", the function that creates HTTP headers in responses allocates an array of iovecs. If that malloc fails, the header creation function returns -1. The function that calls the header-creation function, http_respond, catches that error and itself returns -1. Nothing ever checks the error return from http_respond. (Also, the loop in which memqueue reads entire requests into memory by continuously realloc()'ing a receive buffer has an integer wrap bug in it, though it's probably not triggerable. But incorrect is incorrect, right?)
          • ahh 12y ago
            Also worth noting: under normal conditions on Linux, malloc will never fail--unless you set something like a rlimit on address space, mmap will happily hand you as much address space as you want, then OOM-kill you (or worse, someone else...) when you try to make it resident. So even if we could write malloc-failure-safe software, which we can't, it'd be almost impossible to end up in that condition.