3 ms·
Something which I haven't been able to fully parse that perhaps someone has better insight into: aren't transformers inherently only capable of inductive reason
by mcfry 1y ago
Something which I haven't been able to fully parse that perhaps someone has better insight into: aren't transformers inherently only capable of inductive reasoning? In order to actually progress to AGI, which is being promised at least as an eventuality, don't models have to be capable of deduction? Wouldn't that mean fundamentally changing the pipeline in some way? And no, tools are not deduction. They are useful patches for the lack of deduction.
Models need to move beyond the domain of parsing existing information into existing ideas.
- hammock 1y agoThey can induct just can’t generate new ideas. Its not going to discover a new quark without a human in the loop somewhere
- nightshift1 1y agomaybe that's a good thing after all.
- eli_gottlieb 1y agoThat sounds like a category mistake to me. A proof assistant or logic-programming system performs deduction, and just strapping one of those to an LLM hasn't gotten us to "AGI".
- mcfry 1y agoA proof assistant is a verifier, and a tool so therefor a patch, so I really fail to see how that could be understood as the LLM having deduction.
- energy123 1y agoI don't see any reason to think that transformers are not capable of deductive reasoning. Stochasticity doesn't rule out that ability. It just means the model might be wrong in its deduction, just like humans are sometimes wrong.
- mcfry 1y agoBut it can't actually deduce, can it? If 136891438 * 1294538 isn't in the training data, it won't be able to give you a valid answer using the model itself. There's no process. It has to offload that task to a tool, which will then calculate and return. Further, any offloading needs to be manually defined at some point. You could maybe give it a way to define its own tools, but even then they would still be defined by what has come before.