3 ms·
You can increase the probability of being able to survive an OOM condition long enough to emit diagnostic info in most memory-managed languages (java, python, e
by zbentley 5y ago
You can increase the probability of being able to survive an OOM condition long enough to emit diagnostic info in most memory-managed languages (java, python, etc.) fairly easily. Allocate (and write data to) some global fixed size chunk of ballast memory at program startup, and whenever an allocator failure is detected immediately free that block and emit whatever diagnostics you need. simcop2387 pointed out that Perl even supports this as an interpreter setting; in other languages I've done it manually.
This is more useful than OOM'd programs silently disappearing, but isn't foolproof (that's why I said "increase the probability" rather than "guarantee"): if the OOM killer gets invoked before your telemetry, if you've forked, or if whatever your language does in order to even detect an allocator error is itself memory-costly (looking at you, Python--why on earth would you allocate in order to construct a MemoryError object?!), you will still go down hard.
Other than going back in time and reversing Linux's original sin (allocation success is a lie), swap is the "solution" to these situations, but for many people that cure is worse than the disease.
I often wish there was a generally-available way to map memory to files (rather than mmaping files to memory) selectively in my programs. For cases like this--handling OOM conditions and doing cleanup/reporting before turning out the lights, when the OOM condition was caused either by code in my program outside of my control or other programs on the system outside of my control--having a "all allocations after this instant should occur in a file on disk, not memory" bit to flip would be nice. I don't need my cleanup to be fast; if this is happening I'm already on a sad enough path that I'm happy to trade shutdown/crash performance for fidelity of diagnostic data and increased probability of successful orderly cleanup.
Of course, not being an OS developer, I assume this is impossible for reasons outside of my understanding; the few complicating factors I can think of (copy-on-write pages, for example) are hard enough, and I'm sure there are others.
- jbverschoor 5y agoYeah please don’t.. memory isn’t cheap when everybody does this. This behavior is the reason Java had such a bad time with sysops. Just make use of proper libraries that will invalidate caches if needed, or simply refute to spawn new threads/actors/processes. I’d rather have some application halt/pause or simply crash if it ran out of memory.
- zbentley 5y agoStrongly disagree. First of all, we're talking about miniscule (usually Kb or less) amounts of memory here. Secondly, I'm not talking about over-allocating so programs can run in perpetuity; rather, I'm talking about keeping some small ballast around so cleanup and reporting can happen right before my program crashes/halts (think: the kind of code you'd put in atexit hooks). That's a better experience for operators and product maintainers than "your app disappeared without a trace because the OOM killer came for it" or "your app disappeared without a trace because its ENOMEM behavior was abort()". This is absolutely not why Java had/has a bad rap memory wise--that has more to do with a combination of code that allocates irresponsibly on the happy (not memory-error-anticipating) path, and the JVM's preference to preallocate as much as possible for every purpose, not just OOM handling. > Just make use of proper libraries that will invalidate caches if needed What if I've dumped every cache I can and I'm still getting allocation (or spawn, or whatever) errors because the system is out of memory? It's useful to have a contingency in place to turn out the lights room-by-room rather than cutting power to the whole building, as it were.
- jbverschoor 5y agoMisunderstood.. didn’t see the part that it’s just for handling it. I thought you were talking about just allocation a huge chinch of mem
- zbentley 5y agoNope, just crash ballast. I understand what you're condemning though: it's super regrettable when ordinary utilities, daemons, or desktop apps preallocate like like they're database servers on dedicated hardware.