4 ms·
Because when you're just churning out the most likely set of next tokens, it's hard to understand "I don't know"
by jghn 2y ago
Because when you're just churning out the most likely set of next tokens, it's hard to understand "I don't know"
- AnimalMuppet 2y agoStronger: When you're just churning out the most likely set of next tokens, you don't know anything, in the sense that we normally think of "knowing". The only knowledge that it knows is encoded in the relationship between words. But maybe it could still "not know" in those terms. If there is no next token that is more likely than some threshold, then it "doesn't know" even in "most likely next token" terms.
- SAI_Peregrinus 2y agoThe most likely next tokens could be "I don't know". The output need not have any relation to thought or knowledge.