3 ms·
> if you know about tokenization, you know why this is definitely not a good job for ChatGPT Why are other LLMs able to do it? (Other comments show images suc
by alister 2y ago
> if you know about tokenization, you know why this is definitely not a good job for ChatGPT
Why are other LLMs able to do it? (Other comments show images successfully generated with grok and flux.1)
- sigmoid10 2y agoYou can train the model to do these things in the same way you can train a blind person to describe the colors of objects. But if you put them in an unknown environment or give them a texture they've never encountered before, they will have no idea how to perceive its color. This is a fundamental problem for LLMs and won't change until someone invents a method that gets rid of tokenization for good.