4 ms·
Performance is proportional to the number of reasoning tokens. How to reconcile that with your opinion that they are "random words"?
by energy123 1y ago
Performance is proportional to the number of reasoning tokens. How to reconcile that with your opinion that they are "random words"?
- kelipso 1y agoTechnically random can have probabilities associated with them.. Casual speech, random means equal probabilities, or we don’t know the probabilities. But for LLM token output, it does estimate the probabilities.
- energy123 1y agoGreedy decoding isn't random.
- blargey 1y agos/random/statistically-likely/g Reducing the distance of each statistical leap improves “performance” since you would avoid failure modes that are specific to the largest statistical leaps, but it doesn’t change the underlying mechanism. Reasoning models still “hallucinate” spectacularly even with “shorter” gaps.
- ikari_pl 1y agoWhat's wrong with statistically likely? If I ask you what's 2+2, there's a single answer I consider much more likely than others. Sometimes, words are likely because they are grounded in ideas and facts they represent.
- blargey 1y ago> Sometimes, words are likely because they are grounded in ideas and facts they represent. Yes, and other times they are not. I think the failure modes of a statistical model of a communicative model of thought are unintuitive enough without any added layers of anthropomorphization, so there remains some value in pointing it out.