3 ms·

My understanding in that specific case is that it is a limitation due to the granularity of tokenization. But yes, the major issue is that it simply can not in

My understanding in that specific case is that it is a limitation due to the granularity of tokenization. But yes, the major issue is that it simply can not indicate what it can not do and simply makes up abilities. It is a major limiting factor when we can not know the confidence in correct answers.

xanderlewis 2y ago
I’ve seen lots of people attributing weaknesses (or complete holes) in (autoregressive, transformer-based) LLMs to the tokenisation process, but what exactly does that mean? Surely tokenisation is integral to the whole system? Could you build an LLM that doesn’t use tokens? It feels a bit like someone saying ‘this isn’t a fundamental limitation of cameras; the lens is the problem’… but I’m very much ignorant about these things so I’d be happy to have it explained.
- genrilz 2y ago
  My understanding is that currently several letters are grouped together into tokens for efficiency reasons. For instance, (The) might be a token, as might (cast) and (ing), so an LLM might read "The casting" as "(The)( )(cast)(ing)". Thus it would be quite difficult for such an LLM to answer how often the letter t appears in that sentence unless part of the training text indicated how often t was present in each of its tokens. However, if we didn't care about efficiency, we could make each letter its own token. In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence.
  - xanderlewis 2y ago
    OK, that makes some sense. > In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence. I’m not sure that follows though; is it true that LLMs already find it easy to ‘reason’ about how often (token) appears in some given text? I don’t think so.
    - genrilz 2y ago
      I've only used small local LLMs, which definitely have issues counting and doing math. From what I've seen other commenters do in other HN threads though, it seems like GPT-4 level LLMs do a reasonably good job counting.
  - kgeist 2y ago
    >For instance, (The) might be a token, as might (cast) and (ing), so an LLM might read "The casting" as "(The)( )(cast)(ing) Also, tokens are just numbers internally after tokenization. So LLM sees the question as "how many letters N are there in 563 3684 11267?". It has basically no idea what you're talking about. Unless somewhere in the training data someone mentions that token number 11267 has two letters N.
    - xanderlewis 2y ago
      The fact that tokens are numbers doesn’t seem to have any particular meaning. They’re tokens; they’re elements of a finite set. It doesn’t matter what they ‘are’ because the model doesn’t see this external representation just as it can’t inspect its own architecture. It’s not just the latter part of that sentence that consists of ‘numbers’, all of it does.

xanderlewis 2y ago

I’ve seen lots of people attributing weaknesses (or complete holes) in (autoregressive, transformer-based) LLMs to the tokenisation process, but what exactly does that mean? Surely tokenisation is integral to the whole system? Could you build an LLM that doesn’t use tokens? It feels a bit like someone saying ‘this isn’t a fundamental limitation of cameras; the lens is the problem’… but I’m very much ignorant about these things so I’d be happy to have it explained.

genrilz 2y ago
My understanding is that currently several letters are grouped together into tokens for efficiency reasons. For instance, (The) might be a token, as might (cast) and (ing), so an LLM might read "The casting" as "(The)( )(cast)(ing)". Thus it would be quite difficult for such an LLM to answer how often the letter t appears in that sentence unless part of the training text indicated how often t was present in each of its tokens. However, if we didn't care about efficiency, we could make each letter its own token. In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence.
- xanderlewis 2y ago
  OK, that makes some sense. > In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence. I’m not sure that follows though; is it true that LLMs already find it easy to ‘reason’ about how often (token) appears in some given text? I don’t think so.
  - genrilz 2y ago
    I've only used small local LLMs, which definitely have issues counting and doing math. From what I've seen other commenters do in other HN threads though, it seems like GPT-4 level LLMs do a reasonably good job counting.
- kgeist 2y ago
  >For instance, (The) might be a token, as might (cast) and (ing), so an LLM might read "The casting" as "(The)( )(cast)(ing) Also, tokens are just numbers internally after tokenization. So LLM sees the question as "how many letters N are there in 563 3684 11267?". It has basically no idea what you're talking about. Unless somewhere in the training data someone mentions that token number 11267 has two letters N.
  - xanderlewis 2y ago
    The fact that tokens are numbers doesn’t seem to have any particular meaning. They’re tokens; they’re elements of a finite set. It doesn’t matter what they ‘are’ because the model doesn’t see this external representation just as it can’t inspect its own architecture. It’s not just the latter part of that sentence that consists of ‘numbers’, all of it does.