3 ms·

LLMs are fed token ids, out of a tokenizer.... no characters. They don't even have any concept of a character. You are in a discussion where you are just miles

LLMs are fed token ids, out of a tokenizer.... no characters. They don't even have any concept of a character. You are in a discussion where you are just miles out of your depth. Go read LLMs 101 somewhere.

michaelt 2y ago
If the LLM hasn't learned the letters that comprise input tokens, how do you explain this sort of behaviour? https://chatgpt.com/share/678e95cf-5668-8011-b261-f96ce5a33a5a https://chatgpt.com/share/678e95cf-5668-8011-b261-f96ce5a33a... It can literally spell out words, one letter per line. Seems pretty clear to me the training data contained sufficient information for the LLM to figure out which tokens correspond to which letters. And it's no surprise the training data would contain such content - it'd be pretty easy to synthetically generate misspellings, and being able to deal with typos and OCR mistakes gracefully would be useful in many applications.
- danielmarkbruce 2y ago
  Two answers: 1 - ChatGPT isn't an LLM, its an application using one/many LLMs and other tools (likely routing that to a split function). 2 - even for a single model 'call': It can be explained with the following training samples: "tree is spelled t r e e" and "tree has 2 e's in it" The problem is, the LLM has seen something like: 8062, 382, 136824, 260, 428, 319, 319 and 19816, 853, 220, 17, 319, 885, 306, 480 For a lot of words, it will have seen data that results in it saying something sensible. But it's fragile. If LLMs used character level tokenization, you'd see the first example repeat the token for e in tree rather than tree having it's own token. There are all manner of tradeoffs made in a tokenization scheme. One example is that openai made a change in space tokenization so that it would produce better python code.
HarHarVeryFunny 2y ago
You're the one out of your depth ... LLMs are taught to predict. Once they've seen enough training samples of words being spelled, they'll have learnt that in a spelling context the tokens comprising the word predict the tokens comprising the spelling. Once they've learnt the letters predicted by each token, they'll be able to do this for any word (i.e. token sequence). Of course, you could just try it for yourself - ask an LLM to break a non-dictionary nonsense word like "asdpotyg" into a letter sequence.
- danielmarkbruce 2y ago
  > Once they've learnt the letters predicted by each token, they'll be able to do this for any word (i.e. token sequence). They often fail at things like this, hence the strawberry example. Because they can't break down a token or have any concept of it. There is a sort of sweat spot where it's really hard (like strawberry). The example you give above is so far from a real word that it gets tokenized into lots of tokens, ie it's almost character level tokenization. You also have the fact that none of the mainstream chat apps are blindly shoving things into a model. They are almost certainly routing that to a split function.
  - HarHarVeryFunny 2y ago
    You're still not getting it ... Why would an LLM need to "break down" tokens into letters to do spelling?! That is just not how they work - they work by PREDICTION. If you ask an LLM to break a word into a sequence of letters, it is NOT trying to break it into a sequence of letters - it is trying to do the only thing it was trained to do, which is to predict what tokens (based on the training samples) most likely follow such a request, something that it can easily learn given a few examples in the training set.
    - danielmarkbruce 2y ago
      The LLM can't, thats what makes it relatively difficult. The tokenizer can. Run it through your head with character level tokenization. Imagine the attention calculations. See how easy it would be? See how few samples would be required? It's a trivial thing when the tokenizer breaks everything down to characters. Consider the amount and specificity of training data required to learn spelling 'games' using current tokenization schemes. Vocabularies of 100,000 plus tokens, many of which are close together in high dimensional space but spelled very differently. Then consider the various data sets which give phonetic information as a method to spell. They'd be tokenized in ways which confuse a model. Look, maybe go build one. Your head will spin once you start dealing with the various types of training data and how different tokenization changes things. It screws spelling, math, code, technical biology material, financial material. I specifically build models for financial markets and it's an issue.

michaelt 2y ago

If the LLM hasn't learned the letters that comprise input tokens, how do you explain this sort of behaviour? https://chatgpt.com/share/678e95cf-5668-8011-b261-f96ce5a33a5a https://chatgpt.com/share/678e95cf-5668-8011-b261-f96ce5a33a... It can literally spell out words, one letter per line. Seems pretty clear to me the training data contained sufficient information for the LLM to figure out which tokens correspond to which letters. And it's no surprise the training data would contain such content - it'd be pretty easy to synthetically generate misspellings, and being able to deal with typos and OCR mistakes gracefully would be useful in many applications.

danielmarkbruce 2y ago
Two answers: 1 - ChatGPT isn't an LLM, its an application using one/many LLMs and other tools (likely routing that to a split function). 2 - even for a single model 'call': It can be explained with the following training samples: "tree is spelled t r e e" and "tree has 2 e's in it" The problem is, the LLM has seen something like: 8062, 382, 136824, 260, 428, 319, 319 and 19816, 853, 220, 17, 319, 885, 306, 480 For a lot of words, it will have seen data that results in it saying something sensible. But it's fragile. If LLMs used character level tokenization, you'd see the first example repeat the token for e in tree rather than tree having it's own token. There are all manner of tradeoffs made in a tokenization scheme. One example is that openai made a change in space tokenization so that it would produce better python code.

HarHarVeryFunny 2y ago

You're the one out of your depth ... LLMs are taught to predict. Once they've seen enough training samples of words being spelled, they'll have learnt that in a spelling context the tokens comprising the word predict the tokens comprising the spelling. Once they've learnt the letters predicted by each token, they'll be able to do this for any word (i.e. token sequence). Of course, you could just try it for yourself - ask an LLM to break a non-dictionary nonsense word like "asdpotyg" into a letter sequence.

danielmarkbruce 2y ago
> Once they've learnt the letters predicted by each token, they'll be able to do this for any word (i.e. token sequence). They often fail at things like this, hence the strawberry example. Because they can't break down a token or have any concept of it. There is a sort of sweat spot where it's really hard (like strawberry). The example you give above is so far from a real word that it gets tokenized into lots of tokens, ie it's almost character level tokenization. You also have the fact that none of the mainstream chat apps are blindly shoving things into a model. They are almost certainly routing that to a split function.
- HarHarVeryFunny 2y ago
  You're still not getting it ... Why would an LLM need to "break down" tokens into letters to do spelling?! That is just not how they work - they work by PREDICTION. If you ask an LLM to break a word into a sequence of letters, it is NOT trying to break it into a sequence of letters - it is trying to do the only thing it was trained to do, which is to predict what tokens (based on the training samples) most likely follow such a request, something that it can easily learn given a few examples in the training set.
  - danielmarkbruce 2y ago
    The LLM can't, thats what makes it relatively difficult. The tokenizer can. Run it through your head with character level tokenization. Imagine the attention calculations. See how easy it would be? See how few samples would be required? It's a trivial thing when the tokenizer breaks everything down to characters. Consider the amount and specificity of training data required to learn spelling 'games' using current tokenization schemes. Vocabularies of 100,000 plus tokens, many of which are close together in high dimensional space but spelled very differently. Then consider the various data sets which give phonetic information as a method to spell. They'd be tokenized in ways which confuse a model. Look, maybe go build one. Your head will spin once you start dealing with the various types of training data and how different tokenization changes things. It screws spelling, math, code, technical biology material, financial material. I specifically build models for financial markets and it's an issue.