4 ms·
That's because it uses a long CoT. The actual paper [1] [2] talks about the limitations of decoder-only transformers predicting the reply directly, although it
by orbital-decay 2y ago
That's because it uses a long CoT. The actual paper [1] [2] talks about the limitations of decoder-only transformers predicting the reply directly, although it also establishes the benefits of CoT for composition.
This is all known for a long time and makes intuitive sense - you can't squeeze more computation from it than it can provide. The authors just formally proved it (which is no small deal). And Quanta is being dramatic with conclusions and headlines, as always.
[1] https://arxiv.org/abs/2412.02975 https://arxiv.org/abs/2412.02975
[2] https://news.ycombinator.com/item?id=42889786 https://news.ycombinator.com/item?id=42889786
- leonidasv 2y agoAlso, notice that the original study is from 2023.
- mycall 2y agoThis doesn't include Encoder-Decoder Transformer Fusion for machine translation, or Encoder-Only like text classification, named entity recognition or BERT.
- antirez 2y agoLLMs using CoT are also decoder-only, it's not a paradigm shift like people want to claim now to don't say they were wrong: it's still next token prediction, that is forced to explore more possibilities in the space it contains. And with R1-Zero we also know that LLMs can train themselves to do so.
- teruakohatu 2y agogpt-4o, asked to produce swi-prolog code, gets the same result using a very similar code. gpt4-turbo can do it with slightly less nice code. gpt-3.5-turbo struggled to get the syntax correct but I think with some better prompting could manage it. COT is defiantly optional. Although I am sure all LLM have seen this problem explained and solved in training data.
- janalsncm 2y agoThat’s a different paper than the one this article describes. The article describes this paper: https://arxiv.org/abs/2305.18654 https://arxiv.org/abs/2305.18654