3 ms·
Well if you've hit OOM, you're kinda screwed anyways. But, if you allocate a ring buffer at the beginning, you can always do a best attempt write.
by smj-edison 10mo ago
Well if you've hit OOM, you're kinda screwed anyways. But, if you allocate a ring buffer at the beginning, you can always do a best attempt write.
- koakuma-chan 10mo agoWhy screwed? It could just be that there is more load than your application can handle. Why should it necessarily crash because of that?
- orf 10mo agoYou need to apply backpressure before you hit memory limits, not after. If you’re OOM your application is in a pretty unrecoverable state. Theoretically possible, practically not.
- koakuma-chan 10mo agoIf you allocate a relatively big chunk of memory for each unit of work, and at some point your allocation fails, you can just drop that unit of work. What is not practical?
- ViewTrick1002 10mo agoI think in that case overcommit will happily say the allocation worked. Unless you also zero the entire chunk of memory and then get OOM killed on the write. I suppose you can try to reliable target "seriously wild allocation fails" without leaving too much memory on the table. 0: Heuristic overcommit handling. Obvious overcommits of address space are refused. Used for a typical system. It ensures a seriously wild allocation fails while allowing overcommit to reduce swap usage. root is allowed to allocate slightly more memory in this mode. This is the default. https://www.kernel.org/doc/Documentation/vm/overcommit-accounting https://www.kernel.org/doc/Documentation/vm/overcommit-accou... Running in an environent without overcommit would allow you to handle it gracefully though, although bringing its own zoo of nasty footguns. See this recent discussion on what can happen when turning off overcommit: https://news.ycombinator.com/item?id=46300411 https://news.ycombinator.com/item?id=46300411
- koakuma-chan 10mo ago> See this recent discussion on what can happen when turning off overcommit: What are you referring to specifically? Overcommit is only (presumably) useful if you are using Linux as a desktop OS.
- smj-edison 10mo agoMaybe screwed was too strong of a term. In the scenario above, they wanted to log on resource cleanup, but that makes resource cleanup potentially fallible. The Zig philosophy is that cleanup must never fail, so having cleanup be fallible goes against that. I was suggesting (though in retrospect not clearly) that logging should use a ring buffer instead of allocation, in order to make logging on cleanup a guaranteed best effort operation. You're right that you can recover from OOM, but logging OOM with an allocator is pretty self-defeating.