3 ms·
For NLP tasks it's unlikely that "epistemic uncertainty" is a useful metric, though it's interesting to ask an LLM "what is the third word in this sentence?" an
by voiceblue 2y ago
For NLP tasks it's unlikely that "epistemic uncertainty" is a useful metric, though it's interesting to ask an LLM "what is the third word in this sentence?" and then suggest alternative answers. It's a good way to demonstrate that an LLM is not really a "thinking" machine in the way laymen might assume.
Example:
https://chatgpt.com/share/e8743fe2-a604-4cf0-8c10-aa863a67a59e https://chatgpt.com/share/e8743fe2-a604-4cf0-8c10-aa863a67a5...
- mdp2021 2y ago(Of the two links, I get a 404 and a blank...)
- voiceblue 2y agoThanks, I expected that deleting the conversation on my end would have no effect on a permalink but I was wrong. Nothing I can do about the blank, however, it works for me.
- Manabu-eo 2y agoThere are at least two reasons for transformers poor performance on that prompt: - Transformers view the word as tokens, not words or characters. - The positional encoding might be holding them back. See this recent paper discussed here: Transformers Can Do Arithmetic with the Right Embeddings [1] [1] https://news.ycombinator.com/item?id=40497379 https://news.ycombinator.com/item?id=40497379