3 ms·

Commented this on the blog; cross-posting here: V8 turns out to have a ton of internal string representations. They don’t affect semantics (the general point t

Commented this on the blog; cross-posting here: V8 turns out to have a ton of internal string representations. They don’t affect semantics (the general point that JS string functions think in UTF-16 is valid) but they’re interesting. V8 apparently stores all-ASCII strings as ASCII, so stuff like HTML tag names or base64 blobs doesn’t double in size. Like Go, V8 lets you take a substring as a pointer into the larger string; the internal class is called SlicedString, but from JS-land you don’t see anything different from a string literal. As in Go, keeping a short substring of a long parent string keeps the whole parent ‘alive’ across GCs so sometimes folks will be surprised all those bytes are still allocated. Unlike Go, V8 has a ConsString type, so concatenating strings sometimes doesn’t really immediately copy the underlying bytes anywhere. Building a string with a loop that runs str += newPiece probably goes faster than expected because of this. [It turns out it flattens the string the next time you index into it, or at least used to, which has some perf implications of its own: https://gist.github.com/mraleph/3397008 https://gist.github.com/mraleph/3397008] That’s mostly from this post by a member of the Dart team http://mrale.ph/blog/2016/11/23/making-less-dart-faster.html http://mrale.ph/blog/2016/11/23/making-less-dart-faster.html ; his blog has a lot of interesting stuff about how these fine-tuned language implementations really work. Kind of amazing the lengths V8 (and other JS engine) teams went to to make the code they saw in the wild work well.

evilpie 9y ago
Here is a comment describing strings in SpiderMonkey: http://searchfox.org/mozilla-central/rev/999385a5e8c2d360cc37286882508075fc2078bd/js/src/vm/String.h#44 http://searchfox.org/mozilla-central/rev/999385a5e8c2d360cc3.... If you scroll down there is ASCII art showing the different string sub-classes.
ubernostrum 9y ago
V8 apparently stores all-ASCII strings as ASCII, so stuff like HTML tag names or base64 blobs doesn’t double in size. Python does this too. In Python 2, and in Python 3 until 3.3, a compile-time flag determined the internal Unicode storage of the Python interpreter; a "narrow" build of the interpreter used 2-byte Unicode storage with surrogate pairs, while a "wide" build used 4-byte Unicode storage. As of Python 3.3, the internal storage of Unicode is dynamic. Python 3 source code is always parsed as UTF-8, but then as string objects are created by the interpreter their memory representation is chosen on a per-string basis, to be able to accommodate the widest code point present in the string. So Python will choose either a one-byte, two-byte, or four-byte encoding to store the string in memory, depending on what code points are present in it. This is very nice because it means iteration over a Python string is always iteration over its code points, the length of a string is always the number of code points in it, and indexing always yields the code point at index, since the internal storage of the string is fixed width and never has to include surrogates (in pre-3.3 Python, "narrow" builds would actually yield up things for which ord() gave a value in the surrogate range, and code points requiring surrogates added 2 to the length of a string rather than 1).

evilpie 9y ago

Here is a comment describing strings in SpiderMonkey: http://searchfox.org/mozilla-central/rev/999385a5e8c2d360cc37286882508075fc2078bd/js/src/vm/String.h#44 http://searchfox.org/mozilla-central/rev/999385a5e8c2d360cc3.... If you scroll down there is ASCII art showing the different string sub-classes.

ubernostrum 9y ago

V8 apparently stores all-ASCII strings as ASCII, so stuff like HTML tag names or base64 blobs doesn’t double in size. Python does this too. In Python 2, and in Python 3 until 3.3, a compile-time flag determined the internal Unicode storage of the Python interpreter; a "narrow" build of the interpreter used 2-byte Unicode storage with surrogate pairs, while a "wide" build used 4-byte Unicode storage. As of Python 3.3, the internal storage of Unicode is dynamic. Python 3 source code is always parsed as UTF-8, but then as string objects are created by the interpreter their memory representation is chosen on a per-string basis, to be able to accommodate the widest code point present in the string. So Python will choose either a one-byte, two-byte, or four-byte encoding to store the string in memory, depending on what code points are present in it. This is very nice because it means iteration over a Python string is always iteration over its code points, the length of a string is always the number of code points in it, and indexing always yields the code point at index, since the internal storage of the string is fixed width and never has to include surrogates (in pre-3.3 Python, "narrow" builds would actually yield up things for which ord() gave a value in the surrogate range, and code points requiring surrogates added 2 to the length of a string rather than 1).