3 ms·
It was probably leaking refc binaries, see for example https://ferd.github.io/recon/recon.html#bin_leak-1 https://ferd.github.io/recon/recon.html#bin_leak-1. R
by filmor 4y ago
It was probably leaking refc binaries, see for example https://ferd.github.io/recon/recon.html#bin_leak-1 https://ferd.github.io/recon/recon.html#bin_leak-1.
Running the function (which probably parses large binaries) in a separate process ensures that it's properly garbage collected in time.
- conradfr 4y agoInteresting thanks. Yes that could be it.
- toast0 4y agoRefC binaries should be taken care of by the virtual binary heap, which seems to be from r13 (although I thought it was newer than that... maybe there's another change I'm thinking of, but can't find). A related, but different refc binary hazard is a Process that obtains large refc binaries somehow, and makes a subbinary that it sends to another process (or ets!). The large binary is still referenced from the subbinary so there's a significant amount of excess memory. You can also run into this when binary creation is optimized to allow for appending [1], because that makes a binary of much larger than the required size (either double or 256 bytes, whichever is more). Either way, if you have a use case that naturally results in long term storage of binaries or subbinaries that allocate much more space than is really required, binary:copy/1 can be used to make a clean copy that's the exact size and isn't (yet) shared. I've seen mnesia (ets) nodes where due to the code structure, the memory use ended up at 4x what was needed, and binary:copy added before storing with ets fixed things up with no other code change. [1] https://www.erlang.org/doc/efficiency_guide/binaryhandling.html#constructing-binaries https://www.erlang.org/doc/efficiency_guide/binaryhandling.h...