4 ms·

Two questions: first, how cache friendly is that backing store? and second how the backing store is updated when the strings are garbage collected? At fir

Two questions: first, how cache friendly is that backing store? and second how the backing store is updated when the strings are garbage collected? At first sight it seems that a stack allocated byte sequence can be easily removed but a backing store allocated sequence is not easily handle nor cache friendly. The general problem seems to be about dynamic dispatch of methods, in go using fat pointers, so the author avoid the dynamic dispatch by modifing the code.

tgv 5y ago
> how cache friendly is that backing store It's the original input buffer. It's less fragmented than creating small copies for every token. The backing store stays alive as long as there's a single token pointing to it. It's a bit inefficient if there's a lot of whitespace, comment, etc. in it, or if you're only interested in a very small portion of the tokens, but it doesn't have the overhead of a memory block per string (which is at least 16 bytes). So it depends, I guess. I don't think dynamic dispatch comes into play in go. The strings are represented in the same way.