3 ms·

Naive question: will scaling laws be sufficient for reliable reasoning, or are transformer architectures incapable of that ?

spott 2y ago
Simply scaling to larger models, but retaining naive next token prediction, won’t be enough to get to reliable reasoning on truly novel tasks. Any gains will be reasoning by analogy to a larger number of reasoning examples. But I don’t think any of the big closed source models are relying entirely on next token prediction anymore. They are using reinforcement learning to add new (more complex) objectives to the training. This might allow for better reasoning abilities within the same architecture.
xanderlewis 2y ago
You’re likely to get a few different answers, one of which is some variation on ‘they’re universal function approximations, so of course’. The real answer is: no one could possibly know at this point. My impression is that very few experts believe that transformers alone will lead to AGI (which surely requires ‘reliable reasoning’), even amongst those who believe we’ll all be intellectually replaceable within a decade.
- xanderlewis 2y ago
  > approximations I meant approximators.
astromaniak 2y ago
Alternative question: is being human enough for reliable reasoning? Probably not, but in some, or most, cases may be. You can see it in schools on math exams. Not all pupil can do it, some lack the knowledge, but most simply cannot put things together. "WHY" is another big question.
benreesman 2y ago
It’s not a naive question at all: it’s a very nuanced and controversial question in at least two major ways. The question of what constitutes reasoning is extremely difficult to answer or attempt to answer in a rigorous way. We struggle with clear bright lines on this even as concerns biological organisms with vastly more flexible goal-directed behaviors, and on the closely related concept of consciousness, we lack a consensus even for humans in the womb. Speaking for myself I tend to focus on “useful” levels of planning, generality, and goal-seeking to sidestep some of the thornier philosophical issues. Even there things are wildly controversial. There is a significant group (including some very serious and credentialed experts) who claim that the trajectory is clear: some version of AGI is not only possible with these architectures but so imminent as to demand drastic policy decisions. There is another group (likewise including unimpeachably credentialed experts) who claim that there is no evidence for this extraordinary claim, and that attention decoders in no way show potential for this kind of generality. My understanding of the math and mechanism, for whatever it’s worth, inclines me to agree with the latter group.
- astromaniak 2y ago
  "reliable reasoning" is not the same as AGI. The first can be a big calculator which solves some problems on limited domain. Like engineering, this could boost the progress. Very likely this is possible. Actually it's already happening in some areas like proteins research, not sure it can be called reasoning, though.

spott 2y ago

Simply scaling to larger models, but retaining naive next token prediction, won’t be enough to get to reliable reasoning on truly novel tasks. Any gains will be reasoning by analogy to a larger number of reasoning examples. But I don’t think any of the big closed source models are relying entirely on next token prediction anymore. They are using reinforcement learning to add new (more complex) objectives to the training. This might allow for better reasoning abilities within the same architecture.

xanderlewis 2y ago

You’re likely to get a few different answers, one of which is some variation on ‘they’re universal function approximations, so of course’. The real answer is: no one could possibly know at this point. My impression is that very few experts believe that transformers alone will lead to AGI (which surely requires ‘reliable reasoning’), even amongst those who believe we’ll all be intellectually replaceable within a decade.

xanderlewis 2y ago
> approximations I meant approximators.

astromaniak 2y ago

Alternative question: is being human enough for reliable reasoning? Probably not, but in some, or most, cases may be. You can see it in schools on math exams. Not all pupil can do it, some lack the knowledge, but most simply cannot put things together. "WHY" is another big question.

benreesman 2y ago

It’s not a naive question at all: it’s a very nuanced and controversial question in at least two major ways. The question of what constitutes reasoning is extremely difficult to answer or attempt to answer in a rigorous way. We struggle with clear bright lines on this even as concerns biological organisms with vastly more flexible goal-directed behaviors, and on the closely related concept of consciousness, we lack a consensus even for humans in the womb. Speaking for myself I tend to focus on “useful” levels of planning, generality, and goal-seeking to sidestep some of the thornier philosophical issues. Even there things are wildly controversial. There is a significant group (including some very serious and credentialed experts) who claim that the trajectory is clear: some version of AGI is not only possible with these architectures but so imminent as to demand drastic policy decisions. There is another group (likewise including unimpeachably credentialed experts) who claim that there is no evidence for this extraordinary claim, and that attention decoders in no way show potential for this kind of generality. My understanding of the math and mechanism, for whatever it’s worth, inclines me to agree with the latter group.

astromaniak 2y ago
"reliable reasoning" is not the same as AGI. The first can be a big calculator which solves some problems on limited domain. Like engineering, this could boost the progress. Very likely this is possible. Actually it's already happening in some areas like proteins research, not sure it can be called reasoning, though.