4 ms·

It's _very_ important to know if it failed if you don't want to introduce undefined behavior. With undefined behavior, _anything_ can happen, (e.g. arbitrary co

by _RPM 11y ago

It's _very_ important to know if it failed if you don't want to introduce undefined behavior. With undefined behavior, _anything_ can happen, (e.g. arbitrary code execution). But let me make it clear: malloc failing doesn't mean that undefined behavior will happen. However, pretending like malloc didn't fail when it really did will most likely get you undefined behavior. You're not going to segfault either way if you handle things correctly. Consider a web server running out of memory to allocate a request structure, do you think it is a good idea to just let it SIGSEV?

danieltillett 11y ago
If you have run out of memory there is not much you can really do except crash. Even logging is a problem. Better to just crash and deal with it. I was under the impression that compiler optimisation often removed any NULL checks anyway [1]. 1. http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_14.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-...
- AndyKelley 11y ago
  > I was under the impression that compiler optimisation often removed any NULL checks anyway Goodness. No, you are misunderstanding the example. The optimizer did not introduce a bug by removing a NULL check. The code had a bug because it dereferenced the pointer before checking for NULL.
  - danieltillett 11y ago
    Yes, but they were making the point that thinking you are checking for a null pointer may not always hold especially once inlining happens. Are there any modern platforms where dereferncing a NULL pointer will not cause a crash? Edit. Answering my own question it looks like ARM is one [1]. 1. https://www.securecoding.cert.org/confluence/display/c/EXP34-C.+Do+not+dereference+null+pointers https://www.securecoding.cert.org/confluence/display/c/EXP34...
    - AndyKelley 11y ago
      > thinking you are checking for a null pointer may not always hold especially once inlining happens Incorrect. The example code did not check for a null pointer. It dereferenced a pointer, which implies that the pointer will never be null, and then it pointlessly checked for null on something that could not have been null. The optimizer in fact made the code more clear and easier to read, while maintaining correctness.
      - danieltillett 11y ago
        Quoting from the reference: "While this is intentionally a simple and contrived example, this sort of thing happens all the time with inlining: inlining a function often exposes a number of secondary optimization opportunities. This means that if the optimizer decides to inline a function, a variety of local optimizations can kick in, which change the behavior of the code. This is both perfectly valid according to the standard, and important for performance in practice.” In practice derefencing a NULL pointer on any x86 platform will always cause a crash (defined undefined behaviour I guess), but you can’t assume this on other platforms (ARM seems to be the big one). I have to say before today I had assumed dereferncing a NULL pointer would always cause a segfault. It is always good to learn something new.
- sherjilozair 11y ago
  You can't be further from the truth. A vast magntitude of bugs are due to the system running as if nothing has happened operating on garbage data, and giving you garbage results. If your system is robust to garbage data, say neural network training, good luck finding the bug then.
  - danieltillett 11y ago
    But what do you do in practice if you have run out of memory?
    - davvid 11y ago
      One approach is to free memory, retry, and hope for the best. e.g. https://github.com/git/git/blob/master/wrapper.c#L63 https://github.com/git/git/blob/master/wrapper.c#L63
      - dalke 11y ago
        Another is to pre-allocate enough reserve memory to be able to save and exit cleanly. When malloc fails, switch to the reserve memory and either shut down nicely, or post a dire warning to the user that the end is neigh and to "SAVE AND EXIT IMMEDIATELY".
aivarsk 11y ago
I have some questions: Have you experienced malloc failing a lot? How do you test to ensure error handling code paths work as expected? Have you read 'notes' section of Linux malloc manpage? It says: By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer.
- danieltillett 11y ago
  I have never knowingly experienced malloc failing in any of my code which is why I asked if it is something to worry about in practice. Once you run out of memory things start dying or the system locks up hard. Edit. It is impossible for malloc to fail (at least on linux) [1]. Seems rather pointless checking for something that can’t happen. 1. https://scvalex.net/posts/6/ https://scvalex.net/posts/6/
  - dalke 11y ago
    That link shows that malloc failed, just not due to exceeding available RAM. At the end it says "malloc finally failed because of address space exhaustion."
    - danieltillett 11y ago
      If the memory was actually being used it would have crashed long before that. The test was kind of artificial.
      - dalke 11y ago
        Absolutely. I was pointing out that "It is impossible for malloc to fail" is, according to the linked page, an incorrect summary.
        danieltillett 11y ago

by _RPM 11y ago

danieltillett 11y ago

If you have run out of memory there is not much you can really do except crash. Even logging is a problem. Better to just crash and deal with it. I was under the impression that compiler optimisation often removed any NULL checks anyway [1]. 1. http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_14.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-...

AndyKelley 11y ago
> I was under the impression that compiler optimisation often removed any NULL checks anyway Goodness. No, you are misunderstanding the example. The optimizer did not introduce a bug by removing a NULL check. The code had a bug because it dereferenced the pointer before checking for NULL.
- danieltillett 11y ago
  Yes, but they were making the point that thinking you are checking for a null pointer may not always hold especially once inlining happens. Are there any modern platforms where dereferncing a NULL pointer will not cause a crash? Edit. Answering my own question it looks like ARM is one [1]. 1. https://www.securecoding.cert.org/confluence/display/c/EXP34-C.+Do+not+dereference+null+pointers https://www.securecoding.cert.org/confluence/display/c/EXP34...
  - AndyKelley 11y ago
    > thinking you are checking for a null pointer may not always hold especially once inlining happens Incorrect. The example code did not check for a null pointer. It dereferenced a pointer, which implies that the pointer will never be null, and then it pointlessly checked for null on something that could not have been null. The optimizer in fact made the code more clear and easier to read, while maintaining correctness.
    - danieltillett 11y ago
      Quoting from the reference: "While this is intentionally a simple and contrived example, this sort of thing happens all the time with inlining: inlining a function often exposes a number of secondary optimization opportunities. This means that if the optimizer decides to inline a function, a variety of local optimizations can kick in, which change the behavior of the code. This is both perfectly valid according to the standard, and important for performance in practice.” In practice derefencing a NULL pointer on any x86 platform will always cause a crash (defined undefined behaviour I guess), but you can’t assume this on other platforms (ARM seems to be the big one). I have to say before today I had assumed dereferncing a NULL pointer would always cause a segfault. It is always good to learn something new.
sherjilozair 11y ago
You can't be further from the truth. A vast magntitude of bugs are due to the system running as if nothing has happened operating on garbage data, and giving you garbage results. If your system is robust to garbage data, say neural network training, good luck finding the bug then.
- danieltillett 11y ago
  But what do you do in practice if you have run out of memory?
  - davvid 11y ago
    One approach is to free memory, retry, and hope for the best. e.g. https://github.com/git/git/blob/master/wrapper.c#L63 https://github.com/git/git/blob/master/wrapper.c#L63
    - dalke 11y ago
      Another is to pre-allocate enough reserve memory to be able to save and exit cleanly. When malloc fails, switch to the reserve memory and either shut down nicely, or post a dire warning to the user that the end is neigh and to "SAVE AND EXIT IMMEDIATELY".

aivarsk 11y ago

I have some questions: Have you experienced malloc failing a lot? How do you test to ensure error handling code paths work as expected? Have you read 'notes' section of Linux malloc manpage? It says: By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer.

danieltillett 11y ago
I have never knowingly experienced malloc failing in any of my code which is why I asked if it is something to worry about in practice. Once you run out of memory things start dying or the system locks up hard. Edit. It is impossible for malloc to fail (at least on linux) [1]. Seems rather pointless checking for something that can’t happen. 1. https://scvalex.net/posts/6/ https://scvalex.net/posts/6/
- dalke 11y ago
  That link shows that malloc failed, just not due to exceeding available RAM. At the end it says "malloc finally failed because of address space exhaustion."
  - danieltillett 11y ago
    If the memory was actually being used it would have crashed long before that. The test was kind of artificial.
    - dalke 11y ago
      Absolutely. I was pointing out that "It is impossible for malloc to fail" is, according to the linked page, an incorrect summary.
      - danieltillett 11y ago