5 ms·

> That 64-bit example, however, is IMHO a bug --- I doubt the system he ran it on has anywhere near the 14 exabytes of RAM he asked for, so malloc() succeeding

by spoiler 10y ago

> That 64-bit example, however, is IMHO a bug --- I doubt the system he ran it on has anywhere near the 14 exabytes of RAM he asked for, so malloc() succeeding in that case is ridiculous. Could you elaborate a bit on this? As far as I understand, malloc will succeed if overcommiting is enabled (AFAIK it is in most systems by defualt, but don't hold my word for it since it might be just anecdotal). So, until he actually tries to access the memory—which he never actually does—it won't be allocated. Or is this bollocks?

technion 10y ago
There has been a lot of arguments around not bothering to check the return of malloc(). I can say in this case. char *large = malloc((size_t)-1); if(NULL == large) { perror("Well that failed"); Compiles fine and outputs: Well that failed: Cannot allocate memory
krallin 10y ago
(Note: the following is only applicable to Linux) The default for overcommitting on Linux is heuristic; it doesn't always succeed: if you try and allocate several exabytes or RAM, allocation will definitely fail (in fact, trying to allocate e.g. 2GB of RAM if you only have 1 free will usually fail just the same). There is an option for "always overcommit" (incidentally; the one Redis recommends you use), in which case allocation will always succeed provided the Kernel can represent what you're trying to allocate (what you're describing), but it's definitely not the default Reference: https://www.kernel.org/doc/Documentation/vm/overcommit-accounting https://www.kernel.org/doc/Documentation/vm/overcommit-accou...
- ryao 10y ago
  malloc() should fail even with always over commit enabled if you try to malloc more memory than there is virtual address space, although that needs testing to confirm it. The alternative to failing would be hanging inside the kernel, which would be a DoS attack.
  - krallin 10y ago
    Yes, you're entirely correct; that's actually what I meant by "provided the Kernel can represent [it]" (albeit perhaps not very clearly!). Cheers,
    - ryao 10y ago
      There is also the case of the sum of the allocations of several malloc operations exceeding that threshold. The case of just 1 malloc operation being that large is just a special case of that. That is what I meant, but reading what Ibwrore made me realize that I was not clear about that.
- spoiler 10y ago
  Thanks for clarifying that up! :)

by spoiler 10y ago

technion 10y ago

There has been a lot of arguments around not bothering to check the return of malloc(). I can say in this case. char *large = malloc((size_t)-1); if(NULL == large) { perror("Well that failed"); Compiles fine and outputs: Well that failed: Cannot allocate memory

krallin 10y ago

(Note: the following is only applicable to Linux) The default for overcommitting on Linux is heuristic; it doesn't always succeed: if you try and allocate several exabytes or RAM, allocation will definitely fail (in fact, trying to allocate e.g. 2GB of RAM if you only have 1 free will usually fail just the same). There is an option for "always overcommit" (incidentally; the one Redis recommends you use), in which case allocation will always succeed provided the Kernel can represent what you're trying to allocate (what you're describing), but it's definitely not the default Reference: https://www.kernel.org/doc/Documentation/vm/overcommit-accounting https://www.kernel.org/doc/Documentation/vm/overcommit-accou...

ryao 10y ago
malloc() should fail even with always over commit enabled if you try to malloc more memory than there is virtual address space, although that needs testing to confirm it. The alternative to failing would be hanging inside the kernel, which would be a DoS attack.
- krallin 10y ago
  Yes, you're entirely correct; that's actually what I meant by "provided the Kernel can represent [it]" (albeit perhaps not very clearly!). Cheers,
  - ryao 10y ago
    There is also the case of the sum of the allocations of several malloc operations exceeding that threshold. The case of just 1 malloc operation being that large is just a special case of that. That is what I meant, but reading what Ibwrore made me realize that I was not clear about that.
spoiler 10y ago
Thanks for clarifying that up! :)