5 ms·

I’ve seen lots of people attributing weaknesses (or complete holes) in (autoregressive, transformer-based) LLMs to the tokenisation process, but what exactly do

I’ve seen lots of people attributing weaknesses (or complete holes) in (autoregressive, transformer-based) LLMs to the tokenisation process, but what exactly does that mean? Surely tokenisation is integral to the whole system? Could you build an LLM that doesn’t use tokens? It feels a bit like someone saying ‘this isn’t a fundamental limitation of cameras; the lens is the problem’… but I’m very much ignorant about these things so I’d be happy to have it explained.

genrilz 3y ago
My understanding is that currently several letters are grouped together into tokens for efficiency reasons. For instance, (The) might be a token, as might (cast) and (ing), so an LLM might read "The casting" as "(The)( )(cast)(ing)". Thus it would be quite difficult for such an LLM to answer how often the letter t appears in that sentence unless part of the training text indicated how often t was present in each of its tokens. However, if we didn't care about efficiency, we could make each letter its own token. In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence.
- xanderlewis 3y ago
  OK, that makes some sense. > In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence. I’m not sure that follows though; is it true that LLMs already find it easy to ‘reason’ about how often (token) appears in some given text? I don’t think so.
  - genrilz 3y ago
    I've only used small local LLMs, which definitely have issues counting and doing math. From what I've seen other commenters do in other HN threads though, it seems like GPT-4 level LLMs do a reasonably good job counting.
- kgeist 3y ago
  >For instance, (The) might be a token, as might (cast) and (ing), so an LLM might read "The casting" as "(The)( )(cast)(ing) Also, tokens are just numbers internally after tokenization. So LLM sees the question as "how many letters N are there in 563 3684 11267?". It has basically no idea what you're talking about. Unless somewhere in the training data someone mentions that token number 11267 has two letters N.
  - xanderlewis 3y ago
    The fact that tokens are numbers doesn’t seem to have any particular meaning. They’re tokens; they’re elements of a finite set. It doesn’t matter what they ‘are’ because the model doesn’t see this external representation just as it can’t inspect its own architecture. It’s not just the latter part of that sentence that consists of ‘numbers’, all of it does.

genrilz 3y ago

My understanding is that currently several letters are grouped together into tokens for efficiency reasons. For instance, (The) might be a token, as might (cast) and (ing), so an LLM might read "The casting" as "(The)( )(cast)(ing)". Thus it would be quite difficult for such an LLM to answer how often the letter t appears in that sentence unless part of the training text indicated how often t was present in each of its tokens. However, if we didn't care about efficiency, we could make each letter its own token. In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence.

xanderlewis 3y ago
OK, that makes some sense. > In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence. I’m not sure that follows though; is it true that LLMs already find it easy to ‘reason’ about how often (token) appears in some given text? I don’t think so.
- genrilz 3y ago
  I've only used small local LLMs, which definitely have issues counting and doing math. From what I've seen other commenters do in other HN threads though, it seems like GPT-4 level LLMs do a reasonably good job counting.
kgeist 3y ago
>For instance, (The) might be a token, as might (cast) and (ing), so an LLM might read "The casting" as "(The)( )(cast)(ing) Also, tokens are just numbers internally after tokenization. So LLM sees the question as "how many letters N are there in 563 3684 11267?". It has basically no idea what you're talking about. Unless somewhere in the training data someone mentions that token number 11267 has two letters N.
- xanderlewis 3y ago
  The fact that tokens are numbers doesn’t seem to have any particular meaning. They’re tokens; they’re elements of a finite set. It doesn’t matter what they ‘are’ because the model doesn’t see this external representation just as it can’t inspect its own architecture. It’s not just the latter part of that sentence that consists of ‘numbers’, all of it does.