3 ms·
I've been looking into memory leaks for an Electron app recently, and a major source of problems are strings, strings have sort of this hidden feature/bug: if y
by fabiospampinato 5y ago
I've been looking into memory leaks for an Electron app recently, and a major source of problems are strings, strings have sort of this hidden feature/bug: if you slice a string a new string is not allocated in memory, instead the original string is kept in memory and a couple of numbers indicating the start and the end of the slice as indexes in the parent string are kept in memory. It sounds alright, good for memory usage even, but in many cases you'll be consuming way too much memory because of this.
Example: you split a string into words for spell-checking purposes, once you've spell-checked each word you cache it so that you don't have to ask the spell-checker again if it's valid or not, now each time the original string is edited every single intermediate string produced during the edit could be kept in memory just because you are caching a word out of each of them. This can get out of hand super quickly.
Each time you are extracting a string out of another string this problem can arise.
- adzm 5y agonode has the command line option --nostring_slices to disable this behavior fwiw.
- deleted 5y ago[deleted]
- nolanl 5y agoThat's really interesting. I wonder if this is a Chromium-only issue? It may be worth filing a bug: https://bugs.chromium.org https://bugs.chromium.org I'm not an expert in this space, but it seems to me that the JavaScript engine could eventually clean up that memory, maybe based on some heuristics (e.g. the size of the original string, whether any references to the original string exist, etc.).
- nosianu 5y agohttps://bugs.chromium.org/p/v8/issues/detail?id=2869 https://bugs.chromium.org/p/v8/issues/detail?id=2869 "Issue 2869: Substring of huge string retains huge string in memory" Status: Assigned (Open) Reported 2013
- kohlerm 5y agoSame issue in Java at least for older JDKs. We used to have special (SAP) internal commands to detect this issue in the Eclipse Memory Analyzer.
- jerf 5y agoThis is one of the things that every runtime seems to learn independently, no matter how many times it is learned. I've lost track of the number of languages/runtimes/VMs that started by "cleverly" sharing string memory when you take slices, only to turn the slices back into copies when after a few years a bunch of programs start failing because of tiny little strings holding on to megabyte upon megabyte of underlying string. Another example of how rough this can be is any program speaking over a network because it's really easy to end up with every network packet you've ever gotten (more or less) hanging around in memory because you've got this or that tiny bit of some string that was constructed to represent some chunk of the stream. Surprised to see it in a javascript context, though. I'd have thought they'd have long since passed the point where they found out they have to take copies.
- deleted 5y ago[deleted]
- nosianu 5y agoI ran into this two years ago or so and it was massive - because I extracted small values from HTML microdata stored in files and read into strings, from which I extracted those values. The entire file-contents strings remained in memory. I had an array of a thousand small values extracted from a thousand files, for example, I would end up with an array the size of all those files, not of the extracted values. A gigantic difference for my use case. I ended up writing a hack-string-copy function "substrForceMemCopy" that forced the runtime to allocate a new string by using substr on one less character than I actually wanted and then adding that missing character to force creation of a new string. Main part of the function: return s.substr(start, length - 1) + s.charAt(start + length - 1);