The article uses the phrase 'semantic meaning' three times, but we are dealing with tokens here, which leads me to wonder what sort of semantics sub-word tokens

The article uses the phrase 'semantic meaning' three times, but we are dealing with tokens here, which leads me to wonder what sort of semantics sub-word tokens have. For example, does compositionality [1] apply over sub-word tokens? Does the success of LLMs in generating text that could pass as human-generated suggest that it does? [1] https://en.wikipedia.org/wiki/Principle_of_compositionality https://en.wikipedia.org/wiki/Principle_of_compositionality