The general tokenization screws up performance on some very specific niche tasks, which are not really used in training or evaluation, but are used a 'toy examp

The general tokenization screws up performance on some very specific niche tasks, which are not really used in training or evaluation, but are used a 'toy examples' by people who demo or test the system, resulting in stupid outcomes that are then (IMHO unfairly) treated as informative of the model's general performance. I.e. "how many letters are in a word" or "which is the fifth letter of this word" type tasks is screwed up because the model has no information about the length of each token other than what it can infer from seeing some words in similar challenges in children learning materials; so it knows the length of some tokens (i.e. those used in simple words in similar materials) but not all of them. "does 123912879212392 contain the digit 7" can fail because the model has limited information that the token "79" in the middle is somehow related to the token "7". If you ask whether "87" is contained in that number, then it has to deduce that from seeing tokens "128" and "79", both of which have no direct relation to the digits in them. "is 50100 greater than 50200" becomes a tricky challenge because due to its tokenization it tries to compare "501"/"00" with "50"/"200" while knowing that "501" is greater than "50"; just as comparing "10000" with "100"/"50". We use Arabic numerals because their positional encoding makes arithmetic easier, but language models receive the same data without positional encoding, they get given something that's more like an extreme version of Roman numerals. I mean, all of it can be learned if given sufficient data, however, we shouldn't be surprised if some tasks seem more difficult than they should be if we have explicitly crippled the representation we use to communicate these tasks to the system. And it doesn't neccesarily mean that we should change the tokenization (because these tasks are toy challenges not relevant to most real functioning), we just shouldn't use these types of letter/digit challenges to judge the quality of models.