4 ms·
The question I've never seen them answer correctly is > How many of the letter N does the word "alienation" contain. Mix and match letters and words. It'll ha
by junon 3y ago
The question I've never seen them answer correctly is
> How many of the letter N does the word "alienation" contain.
Mix and match letters and words. It'll hallucinate answers in the largest majority of cases.
What I really want is an LLM that simply tells me "I have no idea sorry." Or some mechanism by which it can provide a confidence score. Until we get there I'm wary of using them for anything other than cursory research or as a fun generation tool.
- 13years 3y agoMy understanding in that specific case is that it is a limitation due to the granularity of tokenization. But yes, the major issue is that it simply can not indicate what it can not do and simply makes up abilities. It is a major limiting factor when we can not know the confidence in correct answers.
- xanderlewis 3y agoI’ve seen lots of people attributing weaknesses (or complete holes) in (autoregressive, transformer-based) LLMs to the tokenisation process, but what exactly does that mean? Surely tokenisation is integral to the whole system? Could you build an LLM that doesn’t use tokens? It feels a bit like someone saying ‘this isn’t a fundamental limitation of cameras; the lens is the problem’… but I’m very much ignorant about these things so I’d be happy to have it explained.
- genrilz 3y agoMy understanding is that currently several letters are grouped together into tokens for efficiency reasons. For instance, (The) might be a token, as might (cast) and (ing), so an LLM might read "The casting" as "(The)( )(cast)(ing)". Thus it would be quite difficult for such an LLM to answer how often the letter t appears in that sentence unless part of the training text indicated how often t was present in each of its tokens. However, if we didn't care about efficiency, we could make each letter its own token. In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence.
- xanderlewis 3y agoOK, that makes some sense. > In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence. I’m not sure that follows though; is it true that LLMs already find it easy to ‘reason’ about how often (token) appears in some given text? I don’t think so.
- genrilz 3y agoI've only used small local LLMs, which definitely have issues counting and doing math. From what I've seen other commenters do in other HN threads though, it seems like GPT-4 level LLMs do a reasonably good job counting.
- kgeist 3y ago>For instance, (The) might be a token, as might (cast) and (ing), so an LLM might read "The casting" as "(The)( )(cast)(ing) Also, tokens are just numbers internally after tokenization. So LLM sees the question as "how many letters N are there in 563 3684 11267?". It has basically no idea what you're talking about. Unless somewhere in the training data someone mentions that token number 11267 has two letters N.
- xanderlewis 3y agoThe fact that tokens are numbers doesn’t seem to have any particular meaning. They’re tokens; they’re elements of a finite set. It doesn’t matter what they ‘are’ because the model doesn’t see this external representation just as it can’t inspect its own architecture. It’s not just the latter part of that sentence that consists of ‘numbers’, all of it does.
- HarHarVeryFunny 3y agoYeah, for the most part using them to transform text (e.g. summarize, translate) rather than generate it from scratch is the way to minimize hallucinations. It's a bit like the difference between thinking when awake, grounded by reality, and dreaming.