3 ms·
> As my other example shows, it can’t handle simple languages like “English backwards.” But that’s not an LLM issue, its a representation model issue. That wou
by dragonwriter 4y ago
> As my other example shows, it can’t handle simple languages like “English backwards.”
But that’s not an LLM issue, its a representation model issue. That would be a simple language variant for a character-based LLM, but its a particularly difficult one for a token-based one, in the same way that certain tasks are more difficult for a human with, say, severe dyslexia.
- kerkeslager 4y agoYes, that's my point: we understand how LLMs work (for example, we know ChatGPT is token-based, not character based), which is what allows us to predict where they'll fail.
- spion 4y agoDescribing things that LLMs can't do isn't proof we understand them. I can describe things that human brains cannot do without understanding how human brains work.
- kerkeslager 4y ago> Describing things that LLMs can't do isn't proof we understand them. That's true, which is why I didn't stop there, but went on to explain why LLMs can't do those things.