4 ms·

Java's sub-strings used to work sort of like this but was changed to use copy semantics. The structure was to have a "char[]" and an "offset" into that array. T

Java's sub-strings used to work sort of like this but was changed to use copy semantics. The structure was to have a "char[]" and an "offset" into that array. This allowed sub-strings to share the underlying array. However if you had a 1 char sub-string to a 1 GB array, the underlying array was never trimmed for garbage collection. In the case of a 1 char substring to a 1 GB string, is Go smart enough to free the rest of the array and keep only the 1 char?

37ef_ced3 6y ago
In Go, as in C, there's no magic. A programmer using Go thinks of a string variable as a pointer/length pair, and knows what will happen. Just like with slices If you keep a pointer into an allocation (in your example, a small Go string pointing into a much larger Go string) the allocation is preserved by the garbage collector You should explicitly copy the substring out (instead of aliasing the underlying string) if retaining the underlying string causes you a problem
- foo_barrio 6y ago
  Okay I understand. In Java, the default String class has the option to "intern" a string which just maintains a list of strings that can be shared. The change was made because often the devs were unaware of the string manipulation taking place in a third party library (eg XML/JSON/HTML parsing). You'd see the memory balloon, investigate and notice that String/char[] instances were dominating your heap. Instead of changing the entire implementation of the standard String class, they changed the semantics of the "substring()" call from O(1) to O(n) + memory side-effects.
KMag 6y ago
I wonder how hard the JVM folks looked into specialized weak references for solving this issue. The mark phase would treat all Strings with zero offset and full length as strongly referencing the byte[], and weakly otherwise. At the end of each full GC, you could iterate over all of your Strings (custom allocate/compact them to their own ranges of the heap for faster scanning), use some heuristics and probabilistic sampling to select some of the weakly reachable byte[]s for size reduction. A specialized copy/compact pass over the Strings could in-place replace byte[] references and fix up offsets. You'd probably also want to modify String.equals() to internally mutate equal strings to point to the same byte[], preferring smaller offsets, and when offsets are equal, preferring lower addresses. This is a light weight lazy version of the background String byte[] interning done by some JVMs.

37ef_ced3 6y ago

In Go, as in C, there's no magic. A programmer using Go thinks of a string variable as a pointer/length pair, and knows what will happen. Just like with slices If you keep a pointer into an allocation (in your example, a small Go string pointing into a much larger Go string) the allocation is preserved by the garbage collector You should explicitly copy the substring out (instead of aliasing the underlying string) if retaining the underlying string causes you a problem

foo_barrio 6y ago
Okay I understand. In Java, the default String class has the option to "intern" a string which just maintains a list of strings that can be shared. The change was made because often the devs were unaware of the string manipulation taking place in a third party library (eg XML/JSON/HTML parsing). You'd see the memory balloon, investigate and notice that String/char[] instances were dominating your heap. Instead of changing the entire implementation of the standard String class, they changed the semantics of the "substring()" call from O(1) to O(n) + memory side-effects.

KMag 6y ago

I wonder how hard the JVM folks looked into specialized weak references for solving this issue. The mark phase would treat all Strings with zero offset and full length as strongly referencing the byte[], and weakly otherwise. At the end of each full GC, you could iterate over all of your Strings (custom allocate/compact them to their own ranges of the heap for faster scanning), use some heuristics and probabilistic sampling to select some of the weakly reachable byte[]s for size reduction. A specialized copy/compact pass over the Strings could in-place replace byte[] references and fix up offsets. You'd probably also want to modify String.equals() to internally mutate equal strings to point to the same byte[], preferring smaller offsets, and when offsets are equal, preferring lower addresses. This is a light weight lazy version of the background String byte[] interning done by some JVMs.