5 ms·
Great tool. BUT, leaks memory like crazy. Despite documentation stating the verbs are fully streaming. > Fully streaming verbs > These don't retain any state
by snidane 4y ago
Great tool.
BUT, leaks memory like crazy.
Despite documentation stating the verbs are fully streaming.
> Fully streaming verbs
> These don't retain any state from one record to the next. They are memory-friendly, and they don't wait for end of input to produce their output.
https://miller.readthedocs.io/en/6.7.0/streaming-and-memory/#fully-streaming-verbs https://miller.readthedocs.io/en/6.7.0/streaming-and-memory/...
- coldtea 4y ago>BUT, leaks memory like crazy. Huh? a) Isn't it written in Golang, which has a GC? Does it do custom buffer based management? b) Isn't it supposed to be run on a file and get some output - as opposed to an interactice session? Why would it matter if it leaks, then, and how could it leak, as the memory is returned to the OS when it ends?
- john_kerl 4y agoIt's indeed written in Go now, which indeed has GC, and ultimately all memory is freed ... but there are indeed some issues around intermediate retention of memory, taking more memory than one would have expected. Some gains were made on https://github.com/johnkerl/miller/pull/1133 https://github.com/johnkerl/miller/pull/1133 and https://github.com/johnkerl/miller/pull/1132 https://github.com/johnkerl/miller/pull/1132