3 ms·

Build perl with -DUSEMYMALLOC and -DPERL_EMERGENCY_SBRK, then you can preallocate a buffer by doing $^M="0" x 65536; then you can trap the out of memory condit

Build perl with -DUSEMYMALLOC and -DPERL_EMERGENCY_SBRK, then you can preallocate a buffer by doing $^M="0" x 65536; then you can trap the out of memory condition with the normal facilities in language and handle it appropriately (mostly letting the big data get deallocated, or exiting). Then you can continue on just like normal. It's a weird setup and I don't think I've run into any other language with that built in.

zbentley 5y ago
Useful, but on Linux it's highly likely that by the time you're comparing $@ to ENOMEM the OOM killer has already awoken and is heading your way.
- Xylakant 5y ago
  The interesting thing is that the OOM killer doesn’t always go for the program that triggered the OOM. It may also decide to kill another memory-hungry process (cough database) on the machine unless you explicitly tweaked it.
  - klyrs 5y ago
    If a program allocates all available memory, and systemd then hits OOM on a 1kB allocation, do you think we should kill systemd?
    - zbentley 5y ago
      I mean, the OOM killer's heuristics are byzantine to be sure. However, if your program is not likely to be the "true" culprit of memory exhaustion, there are better tools at your disposal than ballast pages--cgroups and tunable oomkillers like earlyoom (https://github.com/rfjakob/earlyoom https://github.com/rfjakob/earlyoom). On the other hand, if you are likely to be identified as the culprit, I think the best you can hope for is getting some cleanup/reporting in before you're kill-9'd.
    - Xylakant 5y ago
      I’ve had t he OOM killer kill my work queue because someone wrote a shell script that leaked small amounts of memory. The entire purpose of that machine was to run the queue, so most of the memory was allocated to that process - making it the prime target for the OOM killer. Yes, please, kill everything else on that machine, before touching that process. (Indeed, the OOM killer has a setting for that)
      - klyrs 5y ago
        Sounds bad, but it doesn't justify the proposed heuristic to kill every new process that fails to alloc without looking at the whale that's responsible for the OOM condition. Actually, it sounds like you didn't properly configure that machine. You might be happy to learn that the OOM killer can be configured[1] to specifically protect certain processes. If the entire point of a machine is to run a single process, then you should definitely use that feature. [1] https://lwn.net/Articles/317814/ https://lwn.net/Articles/317814/
    - jamesmishra 5y ago
      I think you've identified the primary exception to the parent comment's rule. Obviously we would not want to kill PID 1, no matter the reason for the OOM.
jbverschoor 5y ago
The next thing OSes will do is use memory compression to go against this behavior. Behold, CS is as bad as governments with all their tax rules

zbentley 5y ago

Useful, but on Linux it's highly likely that by the time you're comparing $@ to ENOMEM the OOM killer has already awoken and is heading your way.

Xylakant 5y ago
The interesting thing is that the OOM killer doesn’t always go for the program that triggered the OOM. It may also decide to kill another memory-hungry process (cough database) on the machine unless you explicitly tweaked it.
- klyrs 5y ago
  If a program allocates all available memory, and systemd then hits OOM on a 1kB allocation, do you think we should kill systemd?
  - zbentley 5y ago
    I mean, the OOM killer's heuristics are byzantine to be sure. However, if your program is not likely to be the "true" culprit of memory exhaustion, there are better tools at your disposal than ballast pages--cgroups and tunable oomkillers like earlyoom (https://github.com/rfjakob/earlyoom https://github.com/rfjakob/earlyoom). On the other hand, if you are likely to be identified as the culprit, I think the best you can hope for is getting some cleanup/reporting in before you're kill-9'd.
  - Xylakant 5y ago
    I’ve had t he OOM killer kill my work queue because someone wrote a shell script that leaked small amounts of memory. The entire purpose of that machine was to run the queue, so most of the memory was allocated to that process - making it the prime target for the OOM killer. Yes, please, kill everything else on that machine, before touching that process. (Indeed, the OOM killer has a setting for that)
    - klyrs 5y ago
      Sounds bad, but it doesn't justify the proposed heuristic to kill every new process that fails to alloc without looking at the whale that's responsible for the OOM condition. Actually, it sounds like you didn't properly configure that machine. You might be happy to learn that the OOM killer can be configured[1] to specifically protect certain processes. If the entire point of a machine is to run a single process, then you should definitely use that feature. [1] https://lwn.net/Articles/317814/ https://lwn.net/Articles/317814/
  - jamesmishra 5y ago
    I think you've identified the primary exception to the parent comment's rule. Obviously we would not want to kill PID 1, no matter the reason for the OOM.

jbverschoor 5y ago

The next thing OSes will do is use memory compression to go against this behavior. Behold, CS is as bad as governments with all their tax rules