4 ms·

It's always funny to read these stories if you know how ChatGPT actually works. Because if you know about tokenization, you know why this is definitely not a go

It's always funny to read these stories if you know how ChatGPT actually works. Because if you know about tokenization, you know why this is definitely not a good job for ChatGPT. Exactly the same reason why it can't spell STRAWBERRY. Not because it doesn't understand the concept of fruits or vegetables or because it doesn't understand sophisticated concepts like metaphors or memes. It's not a good job for it because it doesn't see text in the way you see it. You see the world "hello" made up of individual characters, but the model sees it as a single token (the token with id 24912 for gpt-4o to be precise). It knows the meaning of this token and it's relationship to other tokens much the same way you know relations between words. But it has fundamentally no clue about the characters that make up this word (unless someone trained it to do so or by using spurious additional relations that might exist in the training data).

dools 2y ago
This reminds me of how the alien brain species in Futurama went about gathering all the facts in the Universe. "Beavers mate for life, 11 > 4"
spherelot 2y ago
> But it has fundamentally no clue about the characters that make up this word (unless someone trained it to do so or by using spurious additional relations that might exist in the training data). That was my theory as well when I first saw the strawberry test. However, it is easy test if they know how to spell. The most obvious is: > Can you spell "It is wonderful weather outside. I should go out and play.". Use capital letters, and separate each letter with a space. The free tier ChatGPT model is smart enough to understand the following instructions as well which shows that its not just the simple words: > I was wondering if you can spell. When I ask you a question, answer me with capital letters, and separate each word with a space. When there is real space between the letters, insert character '--' there, so the output is easier to read. Tell me how the attention mechanism works in the modern transformer language models. Also somebody pointed out in some other HN thread that the modern LLMs are perfect for dyslexic people, because you can typo every single word and the model still understands you perfectly. Not sure how true this is, but at least a simple example seems to work: > Hlelo, how aer you diong. Cna you undrestnad me? It would be interesting to know if the datasets actually include spelling examples, or if the models learn how to spell form the massive amount of spelling mistakes in the datasets.
- int_19h 2y ago
  They can do this kind of thing, but in my experience, that makes the model feel "dumber" as far as quality of output goes (unless you make it produce normal output first before having it convert it to something else).
alister 2y ago
> if you know about tokenization, you know why this is definitely not a good job for ChatGPT Why are other LLMs able to do it? (Other comments show images successfully generated with grok and flux.1)
- sigmoid10 2y ago
  You can train the model to do these things in the same way you can train a blind person to describe the colors of objects. But if you put them in an unknown environment or give them a texture they've never encountered before, they will have no idea how to perceive its color. This is a fundamental problem for LLMs and won't change until someone invents a method that gets rid of tokenization for good.
devsda 2y ago
> but the model sees it as a single token (the token with id 24912 for gpt-4o to be precise). It knows the meaning of this token and it's relationship to other tokens much the same way you know relations between words In this context, if we assume that Deep Thought from Hitchhiker's Guide is an LLM, then the answer to everything[1] i.e. 42 makes sense. 42 is just the token id ! 1. https://en.m.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%27s_Guide_to_the_Galaxy#The_Answer_to_the_Ultimate_Question_of_Life,_the_Universe,_and_Everything_is_42 https://en.m.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%...
- kridsdale3 2y ago
  Oh my god, you cracked it. After all these years..
netcan 2y ago
To hazard a silly question... Why can't gpt pick up a non-fundamental "understanding" of letters and spelling from the data? I mean... I do think "see letters" either when speaking/hearing, but I do know how to pull up those letters when necessary.
marcellus23 2y ago
> the token with id 24912 for gpt-4o to be precise How do you find this out?
- MarioMan 2y ago
  OpenAI provides a tokenizer tool: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer
- conesus 2y ago
  GPT-4o uses BPE (byte pair encoding). They released `tiktoken` which allows you to count tokens in strings in python. pip install tiktoken >>> import tiktoken >>> encoding = tiktoken.encoding_for_model("gpt-4o") >>> print(encoding.encode("hello marcellus")) [24912, 2674, 10936, 385]
btown 2y ago
I wonder if there's research being done on training LLMs with extended data in analogy to the "kernel trick" for SVMs: the same way one might feed (x, x^2, x^3) rather than just x, and thus make a linear model able to reason about a nonlinear boundary, should we be feeding multimodal LLMs with not only a token-by-token but also character-by-character and pixel-by-pixel representation of prompt texts during training and inference? Or, allow them to "circle back" and request they be given that as subsequent input text, if they detect that it's relevant information for them? There's likely a lot of interesting work here.

dools 2y ago

This reminds me of how the alien brain species in Futurama went about gathering all the facts in the Universe. "Beavers mate for life, 11 > 4"

spherelot 2y ago

> But it has fundamentally no clue about the characters that make up this word (unless someone trained it to do so or by using spurious additional relations that might exist in the training data). That was my theory as well when I first saw the strawberry test. However, it is easy test if they know how to spell. The most obvious is: > Can you spell "It is wonderful weather outside. I should go out and play.". Use capital letters, and separate each letter with a space. The free tier ChatGPT model is smart enough to understand the following instructions as well which shows that its not just the simple words: > I was wondering if you can spell. When I ask you a question, answer me with capital letters, and separate each word with a space. When there is real space between the letters, insert character '--' there, so the output is easier to read. Tell me how the attention mechanism works in the modern transformer language models. Also somebody pointed out in some other HN thread that the modern LLMs are perfect for dyslexic people, because you can typo every single word and the model still understands you perfectly. Not sure how true this is, but at least a simple example seems to work: > Hlelo, how aer you diong. Cna you undrestnad me? It would be interesting to know if the datasets actually include spelling examples, or if the models learn how to spell form the massive amount of spelling mistakes in the datasets.

int_19h 2y ago
They can do this kind of thing, but in my experience, that makes the model feel "dumber" as far as quality of output goes (unless you make it produce normal output first before having it convert it to something else).

alister 2y ago

> if you know about tokenization, you know why this is definitely not a good job for ChatGPT Why are other LLMs able to do it? (Other comments show images successfully generated with grok and flux.1)

sigmoid10 2y ago
You can train the model to do these things in the same way you can train a blind person to describe the colors of objects. But if you put them in an unknown environment or give them a texture they've never encountered before, they will have no idea how to perceive its color. This is a fundamental problem for LLMs and won't change until someone invents a method that gets rid of tokenization for good.

devsda 2y ago

> but the model sees it as a single token (the token with id 24912 for gpt-4o to be precise). It knows the meaning of this token and it's relationship to other tokens much the same way you know relations between words In this context, if we assume that Deep Thought from Hitchhiker's Guide is an LLM, then the answer to everything[1] i.e. 42 makes sense. 42 is just the token id ! 1. https://en.m.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%27s_Guide_to_the_Galaxy#The_Answer_to_the_Ultimate_Question_of_Life,_the_Universe,_and_Everything_is_42 https://en.m.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%...

kridsdale3 2y ago
Oh my god, you cracked it. After all these years..

netcan 2y ago

To hazard a silly question... Why can't gpt pick up a non-fundamental "understanding" of letters and spelling from the data? I mean... I do think "see letters" either when speaking/hearing, but I do know how to pull up those letters when necessary.

marcellus23 2y ago

> the token with id 24912 for gpt-4o to be precise How do you find this out?

MarioMan 2y ago
OpenAI provides a tokenizer tool: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer
conesus 2y ago
GPT-4o uses BPE (byte pair encoding). They released `tiktoken` which allows you to count tokens in strings in python. pip install tiktoken >>> import tiktoken >>> encoding = tiktoken.encoding_for_model("gpt-4o") >>> print(encoding.encode("hello marcellus")) [24912, 2674, 10936, 385]

btown 2y ago

I wonder if there's research being done on training LLMs with extended data in analogy to the "kernel trick" for SVMs: the same way one might feed (x, x^2, x^3) rather than just x, and thus make a linear model able to reason about a nonlinear boundary, should we be feeding multimodal LLMs with not only a token-by-token but also character-by-character and pixel-by-pixel representation of prompt texts during training and inference? Or, allow them to "circle back" and request they be given that as subsequent input text, if they detect that it's relevant information for them? There's likely a lot of interesting work here.