3 ms·
Yes - some degree of reasoning appears to be latent in the structure of language itself. But models trained explicitly on reasoning-focused data still perform b
by a-t-c-g 6mo ago
Yes - some degree of reasoning appears to be latent in the structure of language itself. But models trained explicitly on reasoning-focused data still perform better than models trained only on general corpora.*
*At least up to 300B parameters, based on the models we’ve tested.
- crawfordcomeaux 6mo agoI wonder what the relationships between the grammar of a language, what it can compute, how it encodes, and what the minimal parameters/structure for reasoning looks like...
- a-t-c-g 6mo agonatural language may provide part of the scaffolding for reasoning, but the capability itself seems to depend more on learned transformations over internal representations than on language alone refs: https://arxiv.org/abs/2412.17819 https://arxiv.org/abs/2412.17819 https://arxiv.org/abs/2412.06769 https://arxiv.org/abs/2412.06769