4 ms·

Tokenization does not remove information from the input[1]. All the information required for character counting is still present in the input following tokeniza

by psb217 2y ago

Tokenization does not remove information from the input[1]. All the information required for character counting is still present in the input following tokenization. The reasons you give for why counting characters is hard could be applied to essentially all other forms of question answering. Ie, to answer questions of type X in general, the LLM will have to generalize from questions of type X in the training corpus to questions of type X with novel surface forms which it sees at test time. [1]tokenizers can remove information if designed to do so, but they don't in these simple scenarios

Al-Khwarizmi 2y ago
As far as I know, that's not the case. The tokenizer takes a bunch of characters, like "berry", identifies it as a token, and what the LLM gets is the token ID. It doesn't have access to the information about which letters that token is composed of. Here is an explanation by OpenAI themselves: https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them https://help.openai.com/en/articles/4936856-what-are-tokens-... - as you can see, "Models take the prompt, convert the input into a list of tokens, processes the prompt, and convert the predicted tokens back to the words we see in the response". And the tokens are basically IDs, without any internal structure - there are examples there. If I'm missing something and you have a source for the claim that character information is present in the input after tokenization, please provide it. I have never implemented an LLM or fiddled with them at low level so I might be missing some detail, but from everything I have read, I'm pretty sure it doesn't work that way.
- psb217 2y ago
  A sequence of tokens can be converted back to the sequence of tokenized characters without loss of information. Eg, how do you think text is rendered for the user based on sequences of tokens generated by the LLM? Different tokenization schemes arrange that information differently and may make it (hand waving here) harder or easier for the model to reason about details like raw character counts that are affected by tokenization. If the training set included sufficiently many examples of character counting Q/A pairs, an LLM would have no trouble learning how to do this task.

by psb217 2y ago

Al-Khwarizmi 2y ago

As far as I know, that's not the case. The tokenizer takes a bunch of characters, like "berry", identifies it as a token, and what the LLM gets is the token ID. It doesn't have access to the information about which letters that token is composed of. Here is an explanation by OpenAI themselves: https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them https://help.openai.com/en/articles/4936856-what-are-tokens-... - as you can see, "Models take the prompt, convert the input into a list of tokens, processes the prompt, and convert the predicted tokens back to the words we see in the response". And the tokens are basically IDs, without any internal structure - there are examples there. If I'm missing something and you have a source for the claim that character information is present in the input after tokenization, please provide it. I have never implemented an LLM or fiddled with them at low level so I might be missing some detail, but from everything I have read, I'm pretty sure it doesn't work that way.

psb217 2y ago
A sequence of tokens can be converted back to the sequence of tokenized characters without loss of information. Eg, how do you think text is rendered for the user based on sequences of tokens generated by the LLM? Different tokenization schemes arrange that information differently and may make it (hand waving here) harder or easier for the model to reason about details like raw character counts that are affected by tokenization. If the training set included sufficiently many examples of character counting Q/A pairs, an LLM would have no trouble learning how to do this task.